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ABSTRACT 

The  mathematical  theory  of  optimal  control/differential  games  is 
used  to  study  the  structure  of  optimal  allocation  policies  for  some  tactical 
allocation  problems  with  combat  described  by  Lanchester-type  equations  of 
warfare.   Both  deterministic  and  stochastic  attrition  processes  are  con- 
sidered.  For  the  optimal  control  of  deterministic  Lanchester-type  attri- 
tion processes,  a  general  solution  algorithm  for  the  synthesis  of  the 
optimal  policy  is  developed.   Optimal  allocation  policies  are  developed 
for  numerous  one-sided  optimization  problems  of  tactical  interest  in  order 
to  study  the  dependence  of  the  structure  of  these  optimal  policies  on  model 
form.   Consideration  has  been  given  to  singular  extremals,  multiple  extremals 
(including  dispersal  surfaces),  and  state  variable  inequality  constraints. 
It  is  shown  how  to  apply  the  theory  of  state  variable  inequality  constraints 
to  determine  the  optimal  control  of  deterministic  Lanchester-type  processes 
in  order  to  treat  non-negativity  restrictions  on  force  levels  and  thus  to 
study  the  dependence  of  optimal  policies  upon  the  force  levels.   Various 
attrition  models  are  considered  (reflecting  different  assumptions  as  to 
target  acquisition  process,  command  and  control  capabilities,  target 
engagement  process,  variations  in  range  capabilities  of  weapon  systems). 
Solutions  are  developed  for  Lanchester-type  equations  of  modern  warfare 
with  variable  attrition-rate  coefficients.   The  optimal  control  of  the 
Lanchester  stochastic  process  is  studied. 

This  task  was  supported  by  the  Naval  Analysis  Programs,  Office  of 
Naval  Research  under  ONR  Project  Order  PO2-0150  and  Task  Number  NR  276-039. 
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PREFACE 

This  report  documents  my  research  on  tactical  allocation  in 
dynamic  combat  situations  over  the  past  two  years  under  the  sponsor- 
ship of  the  Office  of  Naval  Research  (both  as  part  of  the  Foundation 
Research  Program  at  the  Naval  Postgraduate  School  and  under  contracts 
NR  276-027  and  P0  2-0150).   Some  of  this  work  is  recent;  some  of  it 
is  not  so  recent.   In  all  cases,  however,  the  documentation  is  new 
and  does  not  duplicate  any  past  reports. 

Much  of  this  work  has  been  submitted  for  publication  in  the 
open  literature.   However,  because  of  long  "turn  around  times"  for 
refereeing  in  several  journals  (sometimes  over  a  year  before  receiving 
the  referees'  initial  comments),  I  felt  that  it  was  appropriate  to 
include  several  manuscripts  as  appendices  that  had  been  previously 
submitted  for  publication  in  the  open  literature.   Hence,  some  of 
this  work  is  not  of  recent  vintage:   the  paper  "On  the  Solution  to 
Lanchester-Type  Equations  With  Variable  Coefficients"  was  submitted 
for  publication  to  Operations  Research  in  January  1971.   In  many 
instances  I  have  obtained  far  more  extensive  results  but  have  not  had 
the  time  to  fully  document  them. 

Thus,  the  appendices  of  this  report  are  more  or  less  separate 
entities  in  themselves  and  may  be  read  independently  of  each  other. 
Accordingly,  each  appendix  contains  its  own  list  of  references  and 
reference  numbers  apply  only  within  the  appendix  in  which  they  appear. 


A  similar  remark  applies  to  the  numbering  of  equations.   Also,  the 
pagination  is  separate  for  each  individual  appendix. 

The  emphasis  in  this  report  is  on  the  applications  of  control 
theory  to  tactical  allocation  problems  in  dynamic  combat  situations. 
Although  consideration  of  the  various  problems  contained  in  this 
report  has  required  knowledge  of  research  on  the  frontiers  of  deter- 
ministic optimal  control  theory,  I  have  felt  it  to  be  inappropriate 
for  the  scope  of  this  research  to  supply  proofs  for  any  optimization 
theory  results.   However,  I  have  tried  to  give  the  appropriate 
references  to  support  all  statements  of  theory  contained  herein. 
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1.   Introduction. 

This  report  documents  research  performed  under  the  sponsorship 
of  the  Office  of  Naval  Research  (contract  PO  2-0150)  during  the  time 
period  of  27  March  1972  to  16  June  1972.   We  discuss  the  results  of 
a  study  of  some  idealized  models  for  the  optimization  of  combat 
dynamics  and  their  possible  implications  for  defense  planners.   Our 
research  approach  has  been  to  combine  Lanchester-type  formulations 
of  combat  attrition  (both  deterministic  and  stochastic)  and  generalized 
control  theory  [64]  (both  deterministic  and  stochastic  optimal  control, 
dynamic  programming,  differential  games) . 

A  quantitative  theory  of  tactical  allocation  is  developed 
through  the  examination  of  a  sequence  of  simplified  models.   These 
combat  models  are  too  simple  to  be  taken  literally  but  should  be 
interpreted  as  indicating  general  principles  to  serve  as  hypotheses 
for  subsequent  computer  simulation  studies  or  field  experimentation. 
The  effects  of  various  modelling  assumptions  upon  the  structure  of 
the  optimal  allocation  policies  are  systematically  studied  by  con- 
trasting the  solutions  for  various  models. 

A  major  result  of  our  research  is  that  optimal  tactical 
allocation  policies  are  quite  sensitive  to  the  precise  type  of  model 
adopted,  even  as  to  whether  the  tactical  scenario  lasts  a  specified 
period  of  time  or  terminates  only  when  a  pre-determined  state  has 
been  reached.   Insights  are  provided  into  such  important  questions 
as : 


(1)  How  should  targets  be  selected? 

(2)  Do  target  priorities  change  with  time? 

(3)  Do  battle  termination  circumstances  affect  the  optimal 
allocation  policies? 

(4)  How  does  the  nature  of  the  attrition  process  affect  target 
selection? 

(5)  What  is  the  effect  of  ammunition  constraints? 

(6)  How  does  the  uncertainty  and  confusion  of  combat  affect 
the  optimal  selection  rules? 

To  develop  our  theory  of  tectical  allocation  we  have  extensively 

studied  some  specific  combat  scenarios.  Optimal  tactics  for  the 

following  have  been  studied:   selection  of  target  type  at  which  to 

fire,  regulation  of  firing  rate.  The  influences  of  the  following 

factors  have  all  been  considered: 

(1)  combatant  objectives  (form  of  criterion  function  and 
valuation  of  surviving  forces) , 

(2)  weapon  system  performance  characteristics, 

(3)  termination  conditions  of  conflict, 

(4)  force  strengths, 

(5)  type  of  attrition  process, 

(6)  effect  of  resource  constraints, 

(7)  range  capabilities  of  weapon  systems. 

The  tactical  situations  are  described  by  deterministic  Lanchester- 
type  equations  of  warfare.  The  combat  continues  over  a  period  of  time 
with  a  choice  of  tactics  available  to  both  sides  and  subject  to  change 
over  time.   The  mathematical  theory  of  deterministic  optimal  control/ 


differential  games  has  been  used  to  solve  the  problems  under 
consideration.   Thus,  it  seems  appropriate  to  discuss  these  techniques 
briefly. 

a.   Differential  Games/Optimal  Control. 

The  theories  of  optimal  control  and  differential  games  were 
developed  for  optimization  problems  in  which  the  (deterministic) 
system's  dynamics  are  described  by  a  system  of  ordinary  differential 
equations.   The  reader  can  find  references  additional  to  those  given 
here  and  a  brief  review  of  past  developments  in  our  previous  report 
[113]. 

In  a  two-person  zero-sum  deterministic  differential  game  (hence- 
forth abbreviated  to  simply  differential  game)  each  player  chooses 
strategies  (for  precise  definitions,  see  [12]  or  [52])  in  order  to 
maximize  his  own  criterion  functional  (which  when  added  to  that  of 
his  opponent  yields  zero)  for  a  system  whose  dynamics  is  governed  by 
a  system  of  ordinary  differential  equations.   An  example  problem 
relevant  to  our  research  on  tactical  allocation  problems  is 


n  m 

maximize  minimize{  £  w.y.(T)  -  £  v.x.(T)}   with  T  specified, 
*±j<t)    ^.(t)   1=1  X  X      i-1  X  X 


dx.         n 
subject  to: 


dt 


—  =r.  -  >  ib.  ,a.  .y,   for  i  =  1, . . .  ,m, 

i   j=l  1J  iJ  ^ 


dy4         m 

d 


—  =  s.  -  T  <t>..b..x.   for   i  =  1, .  . .  ,n, 
t     i    L   *ij  ij  j 


with 


m 


I   ij;   -  1    for    j  =  l,...,n, 
i=l  J 


n 


I   (j>   »  1   for   j  =  1 m, 

i=l  3 


<f>..,^..  ^  0   and   x.,y.  ^  0,  (1) 

ij  ij  i  i 


where 


x.(t),y.(t)   are  force  levels, 

r.,s.   are  replacement  rates, 

v.,w.   are  the  utilities  assigned  survivors, 

a.,   is  the  rate  at  which  one  Y.   unit  can  destroy  X., 
ij  3  '        i 

b..   is  the  rate  at  which  one  X.   unit  can  destroy  Y., 

<J)..   is  the  fraction  of  X.  who  fire  at  Y., 
ij  J  i 

and   \\> ,  .   is  the  fraction  of  Y.   who  fire  at  X.. 
ij  J  i 

For  our  illustrative  purposes  here,  let  us  assume  that  the  battle  is 
scheduled  to  last  a  prescribed  duration  of  time  (we  further  assume  that 
no  side  is  annihilated  before  T) .   Such  problems  (mainly  pursuit  and 
evasion)  have  been  extensively  studied  in  recent  years.  The  goal  of 
this  research  has  been  to  study  tactical  allocation  problems  using 
such  models . 

Pioneering  research  on  differential  games  was  done  by  R.  Isaacs 
and  culminated  in  his  book  [72] .  Isaacs  developed  a  general  principle 
(the  tenet  of  transition)  in  order  to  develop  his  "main  equation" 


(which  is  basically  a  two-sided  extension  of  the  Hamilton-Jacobi 
equation  (see  [52])).   L.  Berkovitz  [12]  has  justified  in  a  rigorous 
manner  many  of  Isaacs'  original  results,  which  were  heuristically 
obtained.   In  [10]  Berkovitz  developed  many  basic  results  for  differ- 
ential games  according  to  variational  arguments. 

The  above  theories  apply  to  differential  games  which  possess 
solutions  in  pure  strategies.   It  is  a  simple  matter  to  devise  problems 
for  which  this  is  not  true  (see  [11],  [13],  [52]).   However,  Isaacs 
conjectured  that  when  the  Hamiltonian  is  separable,  i.e.  a  function 

independent  of  4»   plus  a  function  independent  of  \p ,      then  the 

* 
differential  game  has  a  solution  in  pure  strategies.   This  conjecture 

was  later  proven  to  be  true  by  L.  Berkovitz  (see  pp.  170-173  of  [10]) 

and  A.  Friedman  [52]  .   We  shall  see  below  that  the  Hamiltonian  for 

(1)  will  be  separable  in  this  sense. 

Let  us  consider  the  above  example  (1)  again.  The  Hamiltonian 

is  given  by 

m         n 
HI 


(t,x,p,<f>,ij;)  =  I   p  {r   -  I   iK  a  y  } 
i=l  x  x        j=i  1J  XJ  J 


n         m 


+   7  q.{s.  -   J"  <f> .  .b.  .x.}, 
1=1  *   1   j=l  1J  1J  J 


(2) 


where 


p  (t)   denotes  the  dual  variable  corresponding  to  x., 
and    q^(t)   denotes  the  dual  variable  corresponding  to  y.. 


We  use  the  symbol  <\>      to  denote  all  the  variables  under  control  of  X. 


In  the  case  when  the  differential  game  has  a  saddle-point  (in  pure 
strategy  solutions)  so  that  a  value  exists,  let  us  denote  the  value 
function  as  W(t,x,y).  This  value  function  is  well-known  [12]  to  be 
C1   (i.e.  continuously  dif ferentiable  with  respect  to  each  of  its 
arguments)  except  at  certain  manifolds  of  discontinuity.   Except  at 
such  manifolds,  one  can  make  the  identification 

1 

and 

<(t)-f-(t). 

J 1 
From  (2) ,  we  see  Hamiltonian  is  indeed  separable  in  the  sense  discussed 
above.  However,  as  not  noted  by  Isaacs,  Berkovitz,  or  Friedman,  the 
above  expression  (2)  for  the  Hamiltonian  function  is  appropriate  only 

when  none  of  the  state  variable  inequality  constraints  (SVIC's)  of  (1) 

* 
is  active.   In  Appendix  E,  we  discuss  the  fact  that  although  problems 

with  SVIC's  have  been  extensively  studied  in  the  literature  of  deter- 
ministic optimal  control  theory  (our  study  of  tactical  allocation 
problems  in  the  Lanchester  theory  of  combat,  however,  has  led  to  the 
uncovering  of  an  important  gap  in  the  existing  theory  of  SVIC's  [115]), 
a  current  gap  in  the  theory  of  differential  games  is  the  lack  of  treat- 
ment of  problems  (such  as  (1)  above)  with  inequality  constraints  on 
functions  of  the  state  variables  only  (the  control  variables  not 


* 

In  (1)   x   for   i  =  1, . . . ,m  and  y .   for  j  =  1, . . . ,n  are  state 

variables,  while  the  <J>'s   and  \|;'s   are  called  control  variables.   An 
example  of  a  SVIC  is  the  condition  that  x^  ^  0,   i.e.  the  X^   force 
level  must  be  non-negative.  Thus,  (2)  is  appropriate  only  for  x.  >  0 
and  y,  >  0  for  all  i. 


explicitly  appearing  in  these  inequality  constraints).   In  fact,  the 
approach  that  R.  Isaacs  used  to  solve  the  "War  of  Attrition  and  Attack" 
(see  pp.  96-104  in  [72])  is  easily  shown  to  be  inadequate  for  other 
similar  problems  (for  further  details  see  Appendix  E) .   (This  important 
gap  in  existing  theory  and  the  inadequacy  of  Isaacs'  (and  others) 
treatment  of  SVIC's  is  not  noted  in  the  recent  work  of  S.  Sternberg 

[111].) 

Thus,  when  x  >  0  and  y,  >  0  for  all  i,   the  Hamiltonian 
is  given  by  (2) .  Hence  the  Hamiltonian  is  separable  (in  the  sense 
discussed  above)  so  that  a  solution  in  pure  strategies  can  be  found 
to  (1) .  Let  us  pursue  further  details  of  solution  development  to 
illustrate  additional  difficulties. 

Assuming  that  x.  >  0  and  y.  >  0  for  all  i,  we  determine 

*  * 
an  extremal  strategy  pair,  denoted  as   (<|>  t\\>   ) ,  by  the  max-min 

principle 

*  *  *  * 

Maximum  minimum  H(t  ,x,p,<J>  ,\J>)  =  H(t,x,p  ,<{>  ,ty   )  .  (3) 

This  leads  to  the  problem 

m  * 

maximize!    )    iJj.  .a.  .  (-p.) } 

i=l     j    XJ 

m 


subject   to:        £  ty    .   =  1 
i=l     J 


ty        2>  0      for     j    =   l,...,n,  (4) 


which  may  be  routinely  solved  to  yield 

'*ij(t>  =  6i,ka 


♦?,e>  - «.,  ,.,.  ^.  (5) 


where 

/  1     for     i  =  j, 

0     otherwise, 

and  k(j,t)   is  the  index  such  that 

akj(_pk^  =  maxlmuin{a1j  (-?]_)»••  •»amj("Pm)l»   for  J  =  1»  •••»n-   (6) 

Similarly  we  obtain 

n  *, 


minimize!    V   <J>..b..q. } 
1  .L,    ii    n    iJ 


1=1   1J    XJ 


subject  to:        £  <J> .  .   =  1 


1-1   1J 


(J)       £  0     for     j   =  l,...,m,  (7) 

* 
so  that  <|>..(t)   is  analogously  determined. 

It  may  be  that  the  index  k(j,t)   is  not  unique,  i.e.  the 

linear  program  (A)  has  alternate  optima.   This  causes  no  difficulty 

unless  this  situation  continues  for  a  finite  interval  of  time.  When 

this  happens,  we  say  that  the  corresponding  segment  of  the  battle 

trajectory  is  a  singular  subarc.   Of  great  practical  significance  is 

the  fact  that  the  max-min  principle  does  not  provide  adequate  tests 


for  whether  a  singular  subarc  can  yield  an  optimal  outcome.  Thus, 
the  possibility  of  singular  subarcs  must  be  investigated  in  tactical 
allocation  differential  games,  since  the  control  (decision)  variables 
appear  linearly  in  problems  such  as  (1) .  We  shall  discuss  this 
important  aspect  further  below  (see  Appendix  D  for  further  details  in 
one-sided  case) ,  but  for  now  let  us  simply  state  that  it  may  be  shown 
that  it  is  not  possible  to  have  a  singular  solution  to  (1) . 

From  (5)  we  see  that  this  model  (1)  yields  that  the  optimal 
tactic  is  to  concentrate  all  fire  on  one  enemy  target  type.   However, 
we  really  don't  gain  that  much  insight  into  changes  in  the  optimal 
target  selection  policy  with  time,  since  we  don't  easily  see  explicitly 
how  the  dual  variables  (and  hence  the  optimal  tactic  via  (4)  and  (7)) 
change  over  time.  The  adjoint  system  of  differential  equations  for 
the  dual  variables  is,  of  course,  given  by 


^■f  2U  *         *         *  !} 


* 


(t,x,p    ,<j>    y\\>   )    =     I  ♦1Hblp  .q,       for      i  =  1,..., 


dt  3x,    *b»-»*    >*    »*   '        .  Vki  ki  k 

i  k=l 


m 


* 

^i  8H  *      *      *  ~      *  * 

dT=  "Wl  (t,x,p  •♦  •*  }  "  jAjakjpk   for   j  =  1'*-"n        (8) 

K.— J. 


with 


p±(t=T)    =  -vi 


qj(«)    -w. 


10 


Hence,  at  the  end  of  battle  at  t  =  T  we  have 


*«<«>-«l.k<J.I>'  ») 


where  k( j ,T)   is  the  index  such  that 

a.  .v  =  maximum{a1  .v  , . .  .  ,a  ,v  },   for  j  =  1,  .  .  .  ,n. 
kj  k  lj  1      mj  m         J     »    »  i 


and 


<!<«>  "  6t,*C3,i)'  C10) 


where  £(j,T)   is  the  index  such  that 

bn .w.  =  maximum{bn  ,wn , . . . ,b  .w  },   for  j  =  l,...,m. 
xj  Jt  lj  1      nj  n         J 

In  Appendix  C,  we  show  how  the  extremal  (candidate  for  optimal 
control)  control  may  be  explicitly  determined  as  a  function  of  time 
for  a  one-sided  analogue  of  the  problem  at  hand. 

Thus ,  we  see  that  for  separable  controls  as  occur  in  target 
selection  problems  in  the  Lanchester  theory  of  combat  the  solution 
method  (i.e.  the  max-min  principle)  essentially  (speaking  somewhat 
imprecisely  now)  reduces  the  problem  of  determining  the  solution  of  the 
differential  game  to  solving  two  "separated"  one-sided  optimization  (optima! 
control  or  variational)  problems.  Moreover,  in  [10]  L.  Berkovitz 
reduces  a  differential  game  to  two  such  control  problems  (problems 


11 


of  Bolza  with  differential  inequalities  as  side  conditions)  in  order 

* 

to  deduce  necessary  conditions  for  optimal  paths. 

It  should  therefore  be  clear  that  if  there  are  difficulties 
(either  theoretical  or  computational)  in  solving  such  optimal  control 
problems,  then  there  will  be  difficulties  in  solving  the  corresponding 
differential  game.   (Alternatively,  we  may  think  of  an  optimal  control 
problem  as  a  differential  game  in  which  the  strategy (s)  of  one  of  the 
two  players  is  fixed  beforehand.)  Accordingly,  our  research  approach 
has  been  to  study  optimal  target  selection  problems  for  dynamic  combat 
situations  with  decisions  available  to  both  sides  by  first  considering 
corresponding  one-sided  (optimal  control)  problems.  After  the  difficult 
points  have  been  mastered  in  the  simple  one-sided  problems,  then  the 
differential  games  are  most  profitably  tackled. 

In  the  next  section  we  discuss  some  aspects  of  deterministic 
optimal  control  problems  that  can  cause  difficulties.  All  these 
subtle  points  may  also  arise  in  differential  games.  We  have  found 
that  by  studying  both  the  theory  and  examples  of  difficult  aspects  in 
one-sided  problems  we  have  appreciably  increased  our  ability  to  solve 
corresponding  differential  games.   In  fact,  as  noted  above,  one  of  our 
research  findings  is  that  a  theory  of  state  variable  inequality  con- 
straints (which  is  essential  in  problems  such  as  (1)  because  of  the 


This  is  done  by  the  famous  method  of  Valentine  (see  [125]).  M.  Hestenes 
[61]  formulated  the  (now  standard)  general  control  problem  and  deduced 
necessary  conditions  (including  the  maximum  principle  (frequently 
referred  to  as  the  Pontryagin  maximum  principle))  according  to  this  pro- 
gram in  1949.  The  same  approach  is  readily  extended  to  nonlinear  program- 
ming (including  linear  programming) ,  but  this  apparently  has  been  over- 
looked by  workers  in  the  field.   However,  our  recent  paper  [118]  remedies 
this  gap. 
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requirement  that  force  levels  be  non-negative)  is  lacking  in  the 
current  theories  for  differential  games.   We  plan  to  develop  such  a 
theory  in  the  future. 

Based  on  this  intimate  relationship  between  the  mathematical 
theories  of  differential  games  and  optimal  control  [63],  our  research 
approach  (as  stated  above)  has  been  to  consider  one-sided  versions  of 
some  tactical  allocation  structures  before  tackling  the  more  realistic 
two-sided  decision  problem.   Our  intent  has  been  to  make  sure  that 
both  theoretical  and  computational  aspects  are  firmly  established  for 
these  optimal  control  problems  before  attempting  to  solve  the  more 
complex  differential  game  versions  of  such  problems. 

Optimal  control  theory  has  emerged  as  a  vigorous  discipline 
since  World  War  II.  Professor  Magnus  Hestenes  of  the  USA  apparently 
first  gave  the  now  standard  control  formulation  for  a  variational 
problem  (as  well  as  developed  optimality  conditions)  and  urged  others 
to  follow  his  approach  in  1949  [61].   Professor  Hestenes  did  not  feel 
that  it  was  appropriate  to  publish  this  work  in  the  open  literature, 
since,  for  example,  his  development  of  a  "Weierstrass  maximum  principle" 
was  a  translation  of  well-known  results  from  the  classical  calculus 
of  variations.   Consequently,  this  pioneering  work  [61]  received  little 


* 
A.  Friedman  (see  Chapter  6  of  [52])  has  developed  a  non-variational 

theory  of  differential  games  via  a  limit  approach.   Although  he  has 
established  that  a  certain  class  of  differential  games  (our  problem  (1) 
belongs  to  this  class)  with  state  variable  inequality  constraints  (SVIC's) 
do  possess  a  value  and  the  optimal  strategies  (see  pp.  238-240  of  [52]) 
does  not  appear  to  be  adequate.   Specifically,  Friedman  makes  an  unsup- 
ported assumption  about  a  property  of  optimal  strategies  (the  assumption 
that  it  is  optimal  to  have  x  (T)  >  0) . 
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attention  from  researchers  other  than  some  at  RAND.   In  1956  Pontryagin 
and  his  associates  announced  their  celebrated  "maximum  principle"  [19] . 
This  event  ushered  in  over  a  decade  of  intense  activity  on  variational 
methods.   Due  to  this  intense  activity  in  the  field  of  optimal  control 
theory  during  the  past  15  years  and  our  specific  interest  in  applica- 
tions to  military  operations  research  problems,  it  does  not  seem 
appropriate  to  cite  here  voluminous  research  articles  (however,  see 
[3]  and  [99]  for  surveys) .   The  current  state  of  the  art  in  deterministic 
optimal  control  theory  is  reflected  in  the  following  contemporary  text- 
books and  monographs  from  which  a  further  more  specialized  guide  to 
the  literature  can  be  obtained  [A],  [6],  [18],  [28],  [40],  [42],  [45], 
[51],  [62],  [89],  [91],  [101]. 

b.   Special  Features  of  Optimal  Control  Problem. 

As  we  have  discussed  above,  our  research  approach  has  been  to 
initially  study  dynamic  tactical  allocation  problems  by  considering 
one-sided  optimal  control  versions  of  them.  Moreover,  these  deter- 
ministic optimal  control  problems  that  we  have  studied  within  the 
framework  of  the  Lanchester  theory  of  combat  have  contained  certain 
special  features  that  have  caused  difficulty  (both  theoretically  and 
computationally) .   Let  us  consider  some  examples  to  illustrate  such 
difficulties. 

For  illustrative  purposes,  we  first  consider  the  following 
optimal  control  problem  for  target  selection  in  Lanchester  combat 
(see  [117]  (reproduced  in  this  report  as  Appendix  A)  for  further  dis- 
cussion of  model) 
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maximize{ry(T)  -  px  (T)  -  qx  (T) }  with  T  unspecified, 
♦  <t) 

dxl 
subject   to:       -r—  =  -<j>any 
J  dt  1 


dx 


£  =  -Vl  -  b2X2  (11) 

0  £  <j>  ^  1   (inequality  constraint  on  control  variable)  , 

x  ,x_,y  ^  0   (state  variable  inequality  constraints), 

where 

x  (t) ,  x  (t) ,  y(t)   are  force  levels, 

p,  q,  r  are  utilities  assigned  survivors, 

a  ,  a  ,  b- ,  b_  are  (constant)  attrition-rate  coefficients, 

and  ^   is  the  fraction  of  Y  fire  directed  at  X  . 

The  stopping  rule  for  the  battle  is  that  the  conflict  terminates 
at  t  =  T  defined  by 


(a)   y(T)  =  0, 
or    (b)   X;L(T)  =  x2(T)  =  0. 


Further  analysis  yields  that  there  are  five  cases  which  must  be 
considered: 
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C±:  Xl(T)  =  0,  x2(T)  >  0,  y(T)  =  0, 

C2:  x1(T)  =  0  before  x2(T)  =  0,  y(T)  >  0, 

C3:  X;L(T)  =  0  after  x2(T)  =  0,  y(T)  >  0, 

C4:  Xl(T)  >  0,  x2(T)  =  0,  y(T)  =  0, 

C5:  X;L(T)  >  0,  x2(T)  >  0,  y(T)  =  0. 

A  constraint  such  as  X.  ^  0  is  referred  to  as  a  state 
variable  inequality  constraint  (SVIC  from  [29]).   When  such  a  constraint 
is  active  (as  when  X- (t)  =  0  for  t.  £  t  £  t?) ,   the  usual  necessary 
conditions  of  optimality  (maXimum  principle)  require  modification: 
the  adjoint  equations  are  modified,  boundary  conditions  for  the  dual 
variables  require  special  treatment,  juncture  (corner)  conditions 
must  be  satisfied,  some  special  multiplier  conditions  must  be  satisfied 
(see  AppendiX  E  for  details) .  This  area  of  deterministic  optimal 
control  theory  (problems  with  SVIC's)  has  had  much  recent  research 
activity  (see  bibliographies  in  [79],  [115]).  However,  the  appropriate 
treatment  of  a  SVIC  in  order  to  determine  the  optimal  trajectory  is 
apparently  not  widely  known  among  practitioners  of  optimal  control 
(as  pointed  out  by  our  recent  note  [115]),  although  the  appropriate 
necessary  conditions  (as  well  as  sufficient  conditions)  are,  of  course, 
well-known  by  active  researchers. 


When  X  ,X  ,y  >  0,   the  Hamiltonian  for  the  problem  (11)  is 


given  by 
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H(t,x,p,<f>)  =  -p1(t){<j)a1y}  -  p2(t){(l-<J))a2y}  -  p^tMb^  +  b^},  (12) 

where  p,  (t)   is  the  dual  variable  corresponding  to  x  ,   etc.   By 
(12)  we  see  that  the  Hamiltonian  is  a  linear  function  of  the  control 
variable  so  that  a  singular  solution  [80],  [81]  is  possible.   Singular 
solutions  to  target  selection  problems  in  the  Lanchester  theory  of 
combat  are  discussed  in  Appendix  D.   This  aspect  of  the  optimal 
control  of  deterministic  Lanchester  attrition  processes  is  of  great 
practical  significance,  since  it  means  that  it  is  possible  to  have 
other  than  <f>   =0  or   1  as  an  optimal  fire  distribution  policy  (in 
contradiction  to  the  results  presented  in  [111]). 

Let  us  further  elaborate  upon  singular  solutions  in  deterministic 
problems  of  optimal  control.   In  such  a  problem,  the  maximum  principle 
may  fail  to  determine  an  optimal  trajectory,  since  the  maximization 
of  the  Hamiltonian  may  not  lead  to  a  well-defined  expression  for 
optimal  control  [80],  [81]  (also  see  Chapter  8  of  [28]).   Singular 
solutions  usually  occur  when  the  Hamiltonian  is  a  linear  function  of 
the  control  variables  (as  is  the  case  for  fire  distribution  problems 
in  the  Lanchester  theory  of  combat  as  formulated  by  Isbell  and  Marlow 
[74]  and  Weiss  (see  pp.  94-95  of  [129])).   However,  all  problems  for 
which  the  Hamiltonian  is  a  linear  function  of  the  control  variables 
do  not  have  singular  subarcs  (see  below)  in  their  solution.   In  par- 
ticular, we  shall  see  that  problem  (11)  does  not. 

The  above  problem  (11)  has  one  control  variable  (denoted  as  <j>)  , 
and  it  appears  linearly  in  the  Hamiltonian.   By  a  singular  subarc  we 
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denote  that  part  of  an  optimal  trajectory  on  which  the  maximum 
principle  cannot  be  used  to  determine  the  control  because  the  coeffi- 
cient of  the  control  variable  in  the  Hamiltonian  is  zero  (see  pp.  226- 
227  of  [80]).   Then  the  term  "singular  solution"  will  be  used  to 
refer  to  any  optimal  trajectory  which  contains  one  or  more  singular 
subarcs . 

To  elaborate  further,  when  the  Hamiltonian  H   is  a  linear 

f)H 
function  of  the  control  variable  d> ,   then  if  —  =  0   for  a  finite 

dtp 

interval  of  time  (or,  another  way  to  say  this,  the  coefficient  of 
cp   vanishes  identically  for  a  finite  interval  of  time)  ,  then  the 
maximum  principle  does  not  determine  the  control.   When  this  situa- 
tion occurs,  neither  the  maximum  principle  nor  the  classical  variational 
theory  provide  adequate  tests  for  maximality  of  the  singular  subarc. 
However,  Kelly  [82],  [83]  generalized  the  Legendre-Clebsch  condition 
of  local  optimality  of  extremal  paths  containing  singular  subarcs 

(see  [75],  [78],  [110]  for  very  recent  developments). 

3H 
Observe  that  on  a  singular  subarc  for  which  —  =  0   for  a 

d(p 

finite  interval  of  time  all  feasible  values  of   <j>  maximize  the 

Hamiltonian.   On  a  singular  subarc  we  determine  the  singular  control 

3H 
by  requiring  that  we  remain  on  the  singular  subarc,  i.e.  —  remains 

dcp 

3H 
zero.   If  — -  is  to  be  identically  equal  to  zero  for  a  finite 

dip 

interval  of  time,  then  all  of  its  derivatives  with  respect  to  time 
must  also  be  equal  to  zero.   We  determine  the  singular  control,  which 
keeps  the  system  on  the  singular  subarc,  by  considering  as  many  of  the 
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3H 
time  derivatives  of  -r—  as  are  required  for  the  control  variable 

4>   to  appear  explicitly  so  that  it  may  be  determined  from  an  algebraic 

equation.   Thus,  in  general  we  consider 

0  a*   dtW   dt*W  c  3) 

We  must  further  check  to  see  that  we  can  get  a  maximum  return 
(in  the  case  when  we  wish  to  maximize  the  criterion  functional)  from 
use  of  the  candidate  singular  subarc.   The  following  condition  (gener- 
alized Legendre-Clebsch  condition)  is  necessary  for  a  singular  subarc 
to  yield  a  maximum  return 


2k 


It  is  obtained  by  examining  the  negative  semidef initeness  of  the 
second  variation  for  a  special  class  of  explicitly  defined  control 
variations  [83] .   For  the  problem  (15)  considered  below,  it  suffices 
to  consider  the  generalized  Legendre-Clebsch  condition  with  k  =  1. 

For  fire  distribution  problems  in  the  Lanchester  theory  of  com- 
bat it  turns  out  that  when  the  rate  of  target-type  attrition  is  pro- 
portional to  the  number  of  enemy  firers  only  (as  in  the  above  problem 


(11)),  it  is  impossible  (excluding  the  pathological  case  when  a-ib-i  = 

A       J^  J-T 

a_b.)  to  have  —  (-rr)=  0.   Thus,  it  is  impossible  to  have  a  singular 
2.    2.  dt  ^a<pj 

* 
solution.   (However,  such  important  aspects  were  not  discussed  in  [111]  ) 


* 
This  is  a  common  gap  in  applications  of  optimal  control  theory  by  many 

practitioners.   For  example,  the  vital  subject  of  singular  solutions  is 

not  discussed  in  [1],  [2],  [67],  or  [106],  even  though  singular  solutions 

are  treated  by  heuristic  means. 
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Our  research  has  yielded  the  fact  that  the  possibility  of 
singular  solutions  must  be  investigated  for  all  such  fire  distribution 
problems  in  the  Lanchester  theory  of  combat.   In  particular,  a  singular 
solution  arises  in  the  following  problem 


maximize{ry(T)  -  px  (T)  -  qx_(T)}  with  T   specified, 
♦  (t) 

dxl 
subject  to:  ^—  =  -<}>a  x  y 


dx2 

IF"  =  "CM)a2x2y 


g  =  -b  xx  -  b2x2  (15) 


0  £  <j>  £  1  and  x  ,x_,y  ^  0. 

In  Appendix  D  we  show  that  it  is  possible  to  have  an  optimal  fire 

distribution  policy  that  is  not  an  extreme  point  in  the  control 

* 
variable  space,  e.g.   <f>   =  a_/(a1+a9).   For  the  problem  at  hand,  this 

explicit  expression  for  the  singular  control  is  obtained  from  the 

equation 

12   faH)    -  n 
TO  "  °- 


dt^9<fr 

^\ -rr  J  ill 

by  use  of  the  conditions  —  =  Trhrr)  =  0  and  the  canonical  equations 

o<p    at  M<Jr 

(i.e.  both  state  and  adjoint  system). 

A  more  subtle  difficulty  in  obtaining  a  complete  solution  to 
fire  distribution  problems  is  that  there  may  be  more  than  one  extremal 
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(an  extremal  is  a  path  on  which  the  necessary  conditions  of  optimality 
(maximum  principle)  are  almost  everywhere  satisfied)  leading  from  an 
initial  point  in  the  force  level  (state)  space  to  the  terminal  surface, 
This  occurs  both  in  the  Isbell-Marlow  problem  [74],  [117]  and  the 
supporting  weapon  system  game  of  H.  K.  Weiss  [130],  [122].   Such  a 
situation  occurs  when  it  is  possible  to  reach  different  points  on 
the  terminal  surface  from  the  same  point  in  the  initial  state  space 
(see  pp.  276-284  of  [4]).   Such  extremals  are  difficult  to  identify, 
since  sufficient  conditions  of  optimality  [53]  may  be  satisfied  along 
each  of  the  multiple  extremals. 

In  problem  (11)  above,  there  are  circumstances  under  which  more 
than  one  extremal  (battle  trajectory  on  which  the  necessary  conditions 
are  satisfied)  leads  from  the  same  initial  force  levels  x  ,  x  ,  yn 
to  the  terminal  surface.   For  example,  under  the  appropriate  circum- 
stances  extremals  may  lead  to  both  C.   and  C^.        Moreover,  it  may 
be  shown  that  sufficient  conditions  of  optimality  [53]  are  satisfied 
for  each  extremal.   However,  this  difficulty  in  solution  development 
is  probably  best  treated  by  example  so  that  we  defer  further  in  depth 
discussion  until  the  appendices. 

To  summarize,  we  have  identified  three  aspects  of  fire  distri- 
bution problems  in  the  Lanchester  theory  of  combat  that  have  not  been 
adequately  treated  previously.  These  aspects  are  as  follows: 


* 
Actually,  there  are  two  types  of  extremals  that  may  lead  to  C_. 
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(1)  state  variable  inequality  constraints, 

(2)  singular  subarcs, 

(3)  multiple  extremals. 

Moreover,  each  of  these  aspects  can  be  vital  in  developing  solutions 
in  applications. 

2.  Objectives  of  Research. 

The  general  objective  of  this  research  is  to  determine  the 
structure  of  the  optimal  fire  distribution/ target  selection  policies 
for  various  specific  scenarios  of  tactical  interest.   The  specific 
objective  of  this  research  on  applications  of  control  theory  to  Lan- 
chester-type  formulations  of  combat  attrition  is  to  determine  the 
sensitivity  of  target  selection  and  force  allocation  strategies  to 
the  form  of  the  attrition  model  and  to  the  time  history  of  force 
levels . 

3.  Related  Work  of  Others. 

Additionally,  we  will  review  in  more  depth  developments  in  the 
quantitative  analysis  of  military  tactics/strategies  through  Lanchester- 
type  models  of  warfare  (i.e.  variational  models  to  determine  optimal 
allocations  (e.g.  target  selection,  fire  distribution,  etc.)  as  a 
function  of  time  and/or  force  levels  in  combat  between  heterogeneous 
forces) .  Hence,  we  consider  the  related  work  of  others  in  the  fields 
of 
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(a)  Lanchester-type  models  of  warfare, 

(b)  differential  games,  and 

(c)  optimal  control  of  Lanchester  attrition  processes. 

We  are  aware  of  probably  well  over  200  titles  of  papers  and 
books  which  contain  information  that,  in  one  sense  or  another,  would 
probably  assist  in  the  attainment  of  the  above  stated  objectives  of 
this  research.   For  the  present  we  shall  highlight  some  major  works. 
In  this  survey  we  consider  the  subject  area  of  differential  games  to 
be  representative  of  applicable  material  within  the  more  extensive 
framework  of  "generalized  control  theory"  (see  [64]  ,  where  this  term 
was  coined  by  Y.  C.  Ho).  More  extensive  reviews  of  pertinent  litera- 
ture are  to  be  found  in  two  of  our  past  reports  [112],  [113]. 

a.  Lanchester-Type  Models  of  Warfare. 

One  of  the  earliest  attempts  to  establish  a  mathematical  model 
of  the  dynamics  of  mass  combat  was  by  Lanchester  [87]  in  1916.   He 
postulated  several  deterministic  models  that  were  a  system  of  ordinary 
differential  equations  which  related  the  strengths  of  opposing  military 
forces  to  length  of  combat.   During  World  War  II  B.  Koopman  extended 
Lanchester 's  results  and  also  suggested  a  reformulation  of  the  problem 
in  stochastic  form  [96].   After  World  War  II  the  RAND  Corporation 
carried  on  further  studies  [60],  [97],  [98]  whose  results  were 
summarized  by  R.  Snow  [109].   H.  K.  Weiss  (then  at  Aberdeen  Proving 
Ground)  and  others  [5],  [26],  [46],  [58],  [59],  [128],  [129]  have 
subsequently  extended  deterministic  Lanchester  models. 
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R.  Brown  developed  models  for  the  stochastic  analysis  of  combat 
[27].  The  relationship  between  the  above  mentioned  stochastic  and 
deterministic  Lanchester  formulations  was  pointed  out  relatively 
early  in  their  development  (see  [109],  for  example)  but  is  probably 
best  discussed  in  a  recent  paper  by  B.  0.  Koopman  [86].  The  first 
probability  analysis  of  Lanchester-type  equations  apparently  appeared 
in  the  (now  classical)  Morse  and  Kimball  book  [96] .  Using  a  stochastic 
model,  an  expression  was  obtained  for  the  probability  of  one  side 
winning  in  a  fight  to  the  finish  in  which  the  random  attrition  process 
corresponds  to  a  deterministic  Lanchester  linear-law  attrition  process. 
Stochastic  and  deterministic  model  results  are  compared.  A  more  in 
depth  comparison  of  stochastic  and  deterministic  model  results  has 
been  given  by  G.  Weiss  [127].   R.  Brown  developed  not  only  expressions 
for  the  probability  of  winning  but  also  approximations  to  the  probability 
of  winning  as  determined  by  stochastic  models  of  combat  attrition 
corresponding  to  both  the  Lanchester  linear  law  and  the  square  law 
[27].   Further  extensions  of  these  results  have  been  given  by  Smith 
[108]  and  Kisi  and  Hirose  [85].  To  date,  the  most  thorough  comparison 
of  deterministic  and  stochastic  homogeneous-force  Lanchester  formula- 
tions has  been  by  G.  Clark  (see  Chapter  11  of  [16]  and  [41]),  who  has 
also  developed  the  time-state  solution  for  the  stochastic  attrition 
process  corresponding  to  a  deterministic  Lanchester  linear-law  process 
[41]. 
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S.  Bonder  [22]  did  the  pioneering  work  on  the  determining  the 
Lanchester  attrition-rate  coefficient  (for  weapon  systems  that  adjust 
fire  based  on  the  results  of  the  immediately  preceding  round).   C. 
Barfoot  [7]  later  suggested  defining  the  Lanchester  attrition-rate 
coefficient  as  the  harmonic  mean  of  the  attrition  rates  (or,  equiva- 
lently,  the  reciprocal  of  the  expected  time  to  kill  a  single  target) . 
Barfoot  also  presented  an  alternate  and  more  general  method  for  com- 
puting the  Lanchester  attrition-rate  coefficient.   Bonder  [23]  then 
showed  that  the  mean  attrition  rate  for  Markov-fire  weapons  is  readily 
obtainable  by  the  methods  described  in  [22]  when  one  defines  the 
Lanchester  attrition-rate  coefficient  as  the  reciprocal  of  the  expected 
time  to  kill  a  single  target.   Kimbleton  [85]  has  extended  the  above 
work  by  deriving  the  distribution  of  the  time  required  to  destroy  a 
target.  Other  papers  [104],  [103]  have  discussed  the  estimation  of 
parameters  for  the  Lanchester  attrition-rate  coefficient. 

A  good  review  of  the  Lanchester  theory  of  combat  is  by  Dolansky 
[46] ,  and  this  includes  a  comprehensive  list  of  references  through 
1965.  A  recent  thesis  directed  by  us  [56]  contains  a  fairly  compre- 
hensive bibliography  of  articles  appearing  in  the  open  literature 
since  1964.   Recent  trends  in  the  Lanchester  theory  of  combat  were 
discussed  at  a  1967  NATO  Conference  [48],  [126].  Of  particular  rele- 
vance to  our  recent  work  is  the  pioneering  work  of  D.  Etter  [49]  (also 
to  appear  in  [48]),  who  has  formulated  and  (partially)  solved  several 
allocation  of  fire  problems  (see  below  for  further  details) .   Etter 
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has  also  given  consideration  to  the  spatial  dimension  of  combat  and 
force  mobility  [49] . 

H.  K.  Weiss  [129]  extended  Lanchester-type  equations  to  include 
the  relative  movement  of  two  homogeneous  forces,  allowing  time  and 
space  to  be  "traded"  for  casualties.  He  considered  the  two  attrition- 
rate  coefficients  to  be  dependent  upon  force  separation  in  such  a  way 
that  their  ratio  was  constant.   S.  Bonder  [20],  [21]  used  Weiss'  extension 
to  study  the  effects  of  mobility  and  various  range  dependencies  for 
the  attrition-rate  coefficients  on  the  number  of  surviving  forces.   He 
has  studied  scenarios  for  combat  between  two  homogeneous  forces,  and 
developed  solutions  for  the  case  of  a  constant  attack  velocity  under 
special  circumstances.   In  our  past  research,  we  have  shown  [114]  that 
a  simple  closed-form  solution  is  possible  in  all  those  cases  when  the 
ratio  of  attrition-rate  coefficients  is  constant,  regardless  of  whether  the 
attack  velocity  is  constant  or  not  (see  also  [116]).  We  have  subse- 
quently developed  [123]  a  completely  general  solution  to  variable 
coefficient  Lanchester-type  equations  for  combat  between  two  homogeneous 
forces  and  further  developed  analytic  results  in  the  following  important 
cases:   (1)  opposing  weapon  systems  whose  kill  rates  have  different 
range  dependencies  and  (2)  weapons  systems  with  different  effective 
ranges.   Some  related  work  appears  in  [24]  and  [43]. 

b.  Differential  Games. 

The  study  of  differential  games  was  initiated  by  R.  Isaacs  at 
RAND  in  the  early  1950's  [68],  [69],  [70],  [71],  but  his  work  has  not 
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been  available  to  a  wide  audience  until  more  recently  [72]  .   His  basic 
concept,  "the  tenet  of  transition,"  is  generalization  of  Bellman's 
[8]  "Principle  of  Optimality"  to  a  competitive  environment,  and  this 
is  used  to  develop  necessary  conditions  for  optimal  strategies.  A 
more  recent  and  more  rigorous  development  of  these  basic  necessary 
conditions  is  by  Berkovitz  [12], 

Since  the  excellent  paper  by  Ho,  Bryson,  and  Baron  [65]  in  1965, 
there  has  been  a  literal  explosion  of  papers  on  differential  games  but 
almost  all  deal  exclusively  with  pursuit-evasion  problems.   Excellent 
survey  papers  which  bear  this  out  are  by  Simakova  (Russian  literature) 
[107]  and  Berkovitz  [13].   (However,  applications  to  the  Lanchester 
theory  of  combat  are  discussed  in  the  next  section.)   Besides  Isaacs' 
book,  others  which  treat  differential  games  are  by  Blaquiere  et  al 
[17]  (extension  of  their  geometric  approach  to  optimal  control) , 
Bryson  and  Ho  [28]  (Chapter  9),  and  Friedman  [52].   This  latter  work 
appeals  to  be  the  most  authoritative  one  on  the  mathematical  theory 
of  differential  games  to  date  and  contains  a  fairly  comprehensive 
bibliography  for  the  purely  mathematical  aspects  of  differential 
games . 

c.   Optimal  Control  of  Lanchester  Attrition  Processes. 

In  1964  Dolansky  [46]  noted  that  an  underdeveloped  area  in  the 
Lanchester  theory  of  combat  was  optimal  target  selection  in  combat 
between  heterogeneous  forces  (optimal  control/differential  games) . 
Our  past  results  [117],  [122]  have  extended  all  the  previously  published 
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results  [74],  [130]  in  the  open  literature  cited  by  Dolansky.  Recently, 
we  have  extended  these  results  even  further  [119],  [120],  [121]. 

In  [117]  we  showed  that  the  solution  originally  obtained  by 
Isbell  and  Marlow  [74]  (also  given  in  [111])  is  incorrect  for  a  cer- 
tain range  of  model  parameters  and  presented  a  general  methodology  for 
solving  all  such  problems.   In  [122]  we  extended  this  methodology  to 
differential  games  by  considering  the  supporting  weapon  system  game 
of  H.  K.  Weiss  [130],  which  was  previously  only  treated  by  heuristic 
means.   (It  was  not  noted  in  [111]  that  Weiss  did  not  apply  the 
mathematical  theory  of  differential  games  to  obtain  his  solution  to 
this  problem.) 

More  recently,  we  have  discussed  the  influence  of  various 
factors  (combatant  objectives,  termination  conditions  of  conflict, 
type  of  attrition  process,  variable  attrition-rate  coefficients,  and 
limited  ammunition)  [119] .   In  [120]  we  presented  some  preliminary 
results  for  optimal  fire  distribution/target  selection  in  combat 
between  heterogeneous  forces  (the  special  case  of  a  homogeneous 
force  against  heterogeneous  enemy  forces) .  We  considered  both  con- 
stant attrition-rate  coefficients  and  some  special  cases  of  variable 
attrition-rate  coefficients.   In  [121]  we  treated  a  simple  problem  of 
target  selection  for  a  homogeneous  force  in  Lanchester  combat  against 
two  enemy  force  types  each  of  which  undergoes  attrition  at  a  rate 
proportional  to  the  product  of  the  numbers  of  firers  and  targets. 
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Some  similar  problems  had  been  previously  studied  by  D.  Etter 
[49]  (also  to  appear  in  [48]).   Although  not  using  modern  optimal 
control  theory,  his  arguments  (which  failed  to  give  consideration  to 
the  long-run  battle  outcome)  did  lead  to  results  which  in  many  cases 
anticipated  many  of  our  recent  results.   Etter  [49]  pointed  out  the 
fundamental  difference  between  the  optimal  allocation  of  fire  with 
square-law  attrition  and  with  linear-law  attrition  and  has  done  some 
computational  studies. 

Work  in  this  area  has  also  been  done  at  the  University  of 
Michigan  [24].   However,  the  dependence  of  optimal  strategies  upon 
model  form  (i.e.  type  of  attrition  process,  battle  termination  condi- 
tions, replacements,  combatant  objectives,  range  dependent  attrition 
rates,  etc.)  was  not  noted  (our  results  are  reported  in  Appendix  C  of 
[113]  and  [117])  and  certain  subtleties  of  optimal  target  selection 
policies  not  detected.  Moreover,  in  [111]  S.  Sternberg  apparently 
gave  the  same  solution  to  a  two-on-one  problem  as  originally  obtained 
by  Isbell  and  Marlow  [74]  and  presented  the  same  solution  development 
methodology  as  they  did.   In  our  past  research  [117]  (reproduced  for 
the  reader's  convenience  in  this  report  as  Appendix  A  (see  also 
Appendix  F))  we  have  shown  that  the  solution  originally  obtained  by 
Isbell  and  Marlow  [74]  is  not  completely  correct.   Hence,  Sternberg's 
claim  (see  p.  153  of  [111])  that  a  "two-on-one  example  was  then  solved 
in  its  entirety"  does  not  seem  justified  to  us. 


29 


In  view  of  the  similarities  in  the  research  areas  considered 
by  Sternberg  [111]  and  the  research  results  presented  in  report,  it 
seems  appropriate  to  make  a  few  critical  comments  about  Sternberg's 
work.   As  we  have  noted  above,  Sternberg  did  not  develop  solution 
methodology  capable  of  obtaining  a  complete  solution  to  the  two-versus- 
one  fire  distribution  problem.   Even  more  serious,  however,  is  his 
failure  to  note  the  well-known  fact  [63],  [65]  that  differential 
games  are  a  class  of  two-sided  optimal  control  problems.   It  is  not 
surprising  then  that  Sternberg  does  not  discuss  the  well-known  control 
theory  results  for  singular  solutions,  state  variable  inequality  con- 
straints (SVIC's),  non-uniqueness  of  extremals,  etc.,  as  applied  to 
differential  games  in  the  Lanchester  theory  of  combat.   Furthermore 
Sternberg  [111]  does  not  note  that  a  theory  for  state  variable 
inequality  constraints  has  not  been  developed  for  differential  games 
or  that  Weiss  [130]  did  not  use  the  necessary  conditions  of  optimality 
to  develop  his  "solution"  to  the  supporting  weapon  system  game. 

The  practical  significance  of  the  above  facts  is  that,  for 
example,  the  solution  of  the  war  of  attrition  and  attack  given  by 
Isaacs  in  [72]  (and  partially  reproduced  by  Sternberg  on  pp.  17-25  of 
[111])  is  not  justified  (the  treatment  of  the  boundary  conditions  for 
the  dual  variables  is  inadequate  as  is  that  for  constrained  subarcs 
when  a  SVIC  is  active) .   (It  is  easy  to  give  an  example  (see  Appendix 
E)  for  which  Isaacs'  approach  does  not  yield  the  correct  solution.) 
Hence,  Sternberg's  attempt  to  solve  a  "two-on-two"  differential  game 
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appears  premature  to  us.   Our  own  research  indicates  that  use  of  the 
appropriate  necessary  conditions  of  optimality  appreciably  reduces 
the  volume  of  allocation  strategy  combinations  that  have  to  be 
considered. 

Additionally,  Sternberg  [111]  did  not  investigate  the  dependence 
of  optimal  allocation  strategies  upon  model  form.   His  major  research 
conclusion  (on  pp.  153-154  of  [111])  that  optimal  allocation  strategies 
in  Lanchester-type  processes  are  always   0,  1   is  incorrect  as  stated, 
since  it  is  easily  shown  that  for  two-versus-one  combat  in  which 
target  types  undergo  attrition  at  rates  proportional  to  the  product  of 
the  numbers  of  firers  and  targets,  the  optimal  allocation  strategy  may 
be  other  than  0  or  1   (see  [121]).   Thus,  Sternberg's  conclusion 
must  be  qualified  be  stating  that  his  results  apply  to,  for  example, 
Lanchester-type  processes  in  which  the  rate  of  attrition  of  each  target 
type  is  proportional  to  (the  sum  over  all  firer  classes  of)  the  number 
of  firers  only  (speaking  somewhat  imprecisely,  Lanchester  square-law 
processes) .   It  was  not  noted  by  Sternberg  that  the  structure  of  the 
optimal  allocation  policy  may  be  different  in  prescribed  duration 
battles  and  fights  to  the  finish. 

Finally,  Bonder  and  Farrell  (see  pp.  27-28  of  [24])  state  that 
(for  Lanchester  attrition  processes  in  which  target  type  attrition  rate 
due  to  each  class  of  firers  is  proportional  to  the  number  of  firers 
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In  Appendix  E  of  this  report  we  develop  a  theory  of  SVIC's  for  one- 
sided fire  distribution  problems  in  the  Lanchester  theory  of  combat. 
Some  of  the  results  we  give  are  not  to  be  found  in  the  literature.   As 
we  have  done  in  the  past  [115],  we  hope  to  publish  some  of  these 
results  in  the  control  theory  literature. 
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only)  the  optimal  target  selection  policy  is  independent  of  the 
number  of  weapons  in  the  firer  or  target  group.   Our  research  results 
(see  [117]  or  Section  9  of  Appendix  G)  contradict  the  above  statement 
made  by  Bonder  and  Farrell:   the  optimal  fire  distribution  policy  may 
depend  indirectly  upon  force  levels. 

Other  similar  tactical/strategic  allocation  problems  that  have 
been  studied  using  differential  game  models  are  the  supporting  weapon 
system  game  of  H.  K.  Weiss  [130],  missile  warfare  (counter-value 
versus  counter-force  targeting)  [105]  ,  and  the  logistics  allocation 
game  of  Moglewer  and  Payne  [95] .  We  have  already  noted  that  Weiss  did 
not  use  the  subsequently  well-known  necessary  conditions  of  optimality 
to  develop  the  solution  in  his  pioneering  work  [130] .  Our  previous 
work  [122]  (see  also  Appendix  B  of  [113])  corrects  this  gap.   Several 
authors  [38],  [39],  [66]  (see  also  pp.  54-57  of  [105])  have  considered 
differential  equation  models  of  a  strategic  missile  exchange  and  have 
attempted  to  study  optimal  tactics  via  differential  game  theory. 
Moglewer  and  Payne  [95]  consider  a  similar  model  for  the  optimal  allo- 
cation of  logistics  resources.  However,  there  are  many  gaps  in  the 
above  work.   For  example,  Moglewer  and  Payne  [95]  consider  (in  our 
terminology  presented  in  [117])  a  terminal  control  differential  game. 
However,  they  fail  to  determine  the  domains  of  controllability  for 
terminal  states  and  express  optimal  tactics  in  terms  of  terminal  force 
levels.   In  [117]  we  showed  that  it  is  essential  to  express  optimal 
tactics  in  terms  of  initial  force  levels. 
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It  seems  appropriate  to  briefly  touch  upon  the  implications 
for  operational  gaming  (see  [100]  or  [124]  for  discussion  of  terminology 
and  background)  of  our  analytic  study  of  tactical  allocation  problems. 
We  believe  that  the  primary  value  of  operational  gaming  is  in  formulat- 
ing the  problem.   (Accordingly,  we  are  coordinating  our  analytic  efforts 
with  the  operational  study  work  of  P.  Chaiken  of  Stanford  Research 
Institute  (Naval  Warfare  Research  Center).)  Moreover,  we  agree  with 
L.  Berkovitz  and  M.  Dresher  in  their  opinion  (see  p.  612  of  [14])  that 
"operational  gaming  is  not  a  helpful  device  for  solving  a  game  or 
getting  significant  information  about  the  solution."  We  believe  that 
insights  into  optimal  allocation  strategies  are  to  be  obtained  through 
analytic  work  such  as  ours. 

Finally,  we  could  find  no  previous  work  on  the  optimal  control 
of  the  Lanchester  stochastic  process  (i.e.  stochastic  versions  of  the 
problems  discussed  above) . 

4.   Overview  of  Research  Program. 

In  this  section  we  briefly  discuss  the  various  subject  areas 
that  we  have  considered  in  developing  our  quantitative  theory  of 
tactical  allocation. 

a.   Combat  Attrition  Models. 

In  our  research  on  tactical  allocation  we  have  given  considera- 
tion to  the  various  different  types  of  models  that  have  been  proposed 
(hypothesized)  in  the  literature  to  describe  combat  between  two  homo- 
heneous  forces.   Such  homogeneous  force  formulations  are  readily 
extended  to  describe  the 
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attrition  for  fire  distribution  problems  in  combat  between  heterogeneous 
forces.   Our  purpose  in  considering  the  various  modelling  alternatives 
for  combat  attrition  has  been  to  determine  the  sensitivity  of  optimal 
target  selection  and  force  allocation  strategies  to  the  form  of  the 
attrition  model.   Based  upon  our  research,  we  conclude  that  optimal 
allocation  policies  are  significantly  affected  by  the  nature  of  the 
attrition  model. 

The  reader  should  recall  that  there  are  two  general  approaches 
which  may  be  taken  to  the  modelling  of  the  attrition  process: 

(a)  deterministic  formulation  which  takes  the  form  of  a  system 
of  first  order  ordinary  differential  equations,  and 

(b)  stochastic  formulation  which  views  the  casualty  process  as 
a  Markov  process . 

In  the  remainder  of  this  section  we  shall  discuss  deterministic  formu- 
lations as  related  to  our  study  of  allocation  strategies.   Stochastic 
combat  attrition  models  are  discussed  in  the  next  section. 

Besides  our  review  of  the  literature  above,  the  interested 
reader  can  find  comprehensive  reviews  in  [41],  [46],  [56].   The  various 
forms  that  the  attrition  rates  can  take  in  models  of  combat  between 
two  homogeneous  forces  has  been  discussed  by  Lanchester  [88] ,  Snow 
[109],  H.  Weiss  [129],  Brackney  [26],  Willard  [131],  and  Helmbold  [59]. 
(The  reader  can  find  a  further  discussion  in  Appendix  H.)   In  our  past 
research  [117],  [120],  [121]  we  have  studied  optimal  target  selection 
policies  when  the  attrition  rate  of  each  target  type  is  proportional 
to: 
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(a)  the  number  of  firers  only, 

(b)  the  product  of  the  numbers  of  firers  and  targets. 

These  correspond  to  the  classical  Lanchester  square-law  process  and 
linear-law  process,  respectively. 

R.  Helmbold  [59]  has  proposed  a  general  formulation  of  combat 
attrition  for  combat  between  two  homogeneous  forces  (based  upon  his 
empirical  study  of  a  large  number  of  land  battles  [57]).   His  general 
model  [59]  accounts  for  inefficiencies  of  scale  when  force  sizes  are 
grossly  unequal.   In  this  report  we  present  some  preliminary  results 
for  optimal  target  selection  when  Helmbold' s  general  model  is  used 
to  describe  attrition  in  combat  between  heterogeneous  forces.   In 
doing  this  work  we  have  extended  his  formulation  to  combat  between 
heterogeneous  forces  and  present  some  analytic  solutions. 

b.   Deterministic  Versus  Stochastic  Models  of  Combat  Attrition. 

As  indicated  by  our  review  of  the  pertinent  literature  and 
noted  above,  two  basic  approaches  to  analytic  war  gaming  are  (1)  the 
Lanchester-type  deterministic  differential  equation  models   [24],  [25], 
[50]  or  (2)  stochastic  models  (such  as  DYNTACS  [16],  a  high  resolution 

simulation) .   Almost  all  previously  published  research  on  tactical 

** 

allocation  problems   (including  our  own  work)  has  treated  attrition 

as  a  deterministic  process.   We  have  briefly  discussed  work  on  stochastic 
formulations  in  the  review  of  pertinent  literature  above.   In  Section  2 


These  are  frequently  referred  to  as  expected -value  models. 

** 

The  one  exception  of  a  stochastic  optimization  problem  is  the  work 
of  Isbell  and  Marlow  [73]. 
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of  Appendix  I  the  reader  can  find  a  more  comprehensive  review  of  past 
work  on  the  Lanchester  stochastic  process.  There  are,  however,  con- 
flicting views  in  the  literature  as  to  whether  these  two  approaches 
yield  the  same  results.   (We  certainly  don't  plan  to  try  to  settle 
this  difficult  question,  but  wish  to  caution  the  reader  that  there 
is  not  universal  agreement  among  workers  in  the  field  about  this 
important  question.) 

G.  Clark  has  studied  (see  Chapter  11  of  [16]  and  [41])  the  differ- 
ence between  deterministic  and  stochastic  homogeneous-force  Lanchester 
formulations  and  concludes  that  (see  p.  11-19  of  [16])  "the  determin- 
istic model  would  have  difficulty  approximating  a  stochastic  simulation" 
with  respect  to  the  time  history  of  force  levels.   Based  upon  our  study 
of  Clark's  work  [41],  we  raise  the  question  as  to  whether  there  may 
be  significant  differences  between  optimal  allocation  policies  for 
deterministic  and  stochastic  attrition  processes.   If  such  a  difference 
were  to  be  observed,  then  this  would  raise  the  more  basic  question  of 
whether  or  not  different  study  results  could  be  obtained  merely  by 
the  choice  of  modelling  technique,  i.e.  deterministic  or  stochastic 
formulations. 

c.   Special  Features  of  Deterministic  Optimal  Control  Models 
for  Tactical  Allocation. 

In  Section  1.  above,  we  have  discussed  three  important  problem 
areas  in  applying  deterministic  optimal  control  theory  to  study  tactical 
allocation  problems 
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(a)  non-uniqueness  of  extremals, 

(b)  state  variable  inequality  constraints, 

(c)  singular  subarcs. 

(These  topics  were  not  discussed  in  [111].)   Most  of  our  research 
effort  has  been  directed  at  developing  a  satisfactory  treatment  of 
these  topics  for  tactical  allocation  problems.  All  are  relatively 
near  the  frontiers  of  research  in  deterministic  optimal  control  theory. 

Further  detailed  discussions  of  the  above  topics  can  be  found 
in  the  appropriate  appendices  which  are  as  follows : 

(a)  non-uniqueness  of  extremals:   Appendices  A,  F,  G, 

(b)  state  variable  inequality  constraints:  Appendix  E, 

(c)  singular  subarcs:  Appendix  D. 

It  seems  appropriate,  however,  to  give  further  general  discussion  of 
these  aspects  in  order  to  emphasize  their  importance  in  the  optimiza- 
tion of  combat  dynamics  via  optimal  control  theory.  Additionally,  we 
will  review  their  current  status  within  deterministic  optimal  control 
theory. 

(1)   Non-uniqueness  of  Extremals. 
The  purpose  of  this  research  is  to  determine  optimal  fire  dis- 
tribution/target selection  policies  for  tactical  allocation  problems. 
However,  as  is  well-known,  the  maximum  principle  (or  the  max-min 
principle  for  two-sided  (differential  game)  problems)  provides  only 
necessary  conditions  of  optimality.   Moreover,  let  the  reader  recall 
that  we  refer  to  a  path  on  which  the  necessary  conditions  of  optimality 
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(maximum  printiple)  are  almost  everywhere  satisfied  as  an  extremal . 
Thus,  in  order  to  determine  the  optimal  allocation  policy  one  must 
show  that  a  particular  extremal  is  indeed  an  optimal  trajectory 
(otherwise  one  may  incorrectly  identify  a  non-optimal  policy  as  being 
optimal) . 

Two  ways  of  determining  optimal  trajectories  are  as  follows: 

(a)  check  whether  sufficient  conditions  of  optimality  [92],  [53] 
are  satisfied  on  the  extremal, 

** 

(b)  by  citing  the  appropriate  existence  theorem,  show  that  an 

optimal  control  exists  to  the  problem  at  hand;  there  are  two 
further  subcases:   (1)  if  the  extremal  is  unique,  then  it  is 
optimal  or  (2)  if  the  extremal  is  not  unique  and  only  a 
finite  number  exist,  then  the  optimal  trajectory  is  determined 
by  considering  a  finite  number  of  alternatives. 

It  turns  out,  however,  that  the  former  approach  is  inadequate  for  most 

tactical  allocation  problems  that  we  have  considered,  since  existing 

sufficient  conditions  [92],  [53]  only  apply  to  optimal  control  problems 

of  fixed  length  (i.e.  fixed  terminal  time).  Without  going  into  subtle 

details  at  this  time,  let  us  state  that  the  results  of  [92],  [53]  cannot 

be  applied  to,  for  example,  the  Isbell-Marlow  problem  [74],  [117] 

without  qualification  (if  they  are,  it  is  easy  to  falsely  conclude 

that  certain  non-optimal  extremals  are  optimal) . 

The  second  approach  given  above  is  the  one  we  have  taken  in  this 

research.   If  an  extremal  is  unique,  then  it  is  optimal  and  no  difficulty 


For  the  purposes  of  our  expository  discussion  here  we  are  speaking 
somewhat  imprecisely  in  order  to  communicate  to  the  general  reader  why 
certain  topics  have  received  so  much  attention  in  our  research. 

** 

This  is  essentially  a  condensation  of  the  general  solution  procedure 

given  in  Appendix  A  (and  further  extended  in  Appendix  G) . 
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exists.   However,  in  the  simplest  fire  distribution  problem  we  have 
shown  that  extremals  may  not  be  unique  [117].   (Apparently,  this  was 
not  noted  in  any  past  research  on  this  problem  [74],  [111].)   Moreover, 
we  have  shown  how  to  determine  the  optimal  allocation  policy  when 
there  are  multiple  extremals  in  such  a  problem  (see  Appendices  A  and 
F  for  further  details) .   In  the  latter  case,  one  may  have  to  contend 
with  a  dispersal  surface  (see  pp.  132-141  of  [72])  being  present  in 
the  solution. 

In  our  past  research,  we  have  shown  that  dispersal  surfaces  may 
exist  in  the  solution  to  even  the  simplest  fire  distribution  problem 
[117].   This  rare  singular  surface  is  very  difficult  to  determine,  since 
its  presence  in  a  solution  cannot  be  determined  directly  by  the  maximum 
principle  but  requires  "considerations  in  the  large."  Besides  the 
examples  (all  two-sided  problems)  given  in  Isaacs'  book  (see  pp.  135- 
155  of  [72]),  we  are  aware  of  only  one  other  problem  [44]  studied  in 
the  literature  in  whose  solution  a  dispersal  surface  has  been  found. 

The  existence  of  multiple  (i.e.  non-uniqueness  of)  extremals  in 
tactical  allocation  problems  within  the  Lanchester  theory  of  combat 
has  important  implications  for  computational  methods  (such  as  dynamic 
programming  or  other  finite  difference  approximation  techniques) . 
Approximate  solutions  to  optimal  control  problems  may  be  developed 
by  discretizing  time  (i.e.  using  finite  difference  approximations) 
and  treating  the  resulting  problem  as  a  mathematical  programming  problem 
[30],  [93].   The  difficulties  caused  by  the  existence  of  multiple  extremals 
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or  dispersal  surfaces  in  the  solution  to  a  continuous-time  optimal 
control  problem  are  not  circumvented  by  considering  an  approximating 
mathematical  programming  (finite-dimensional  optimization)  problem. 

With  such  subtle  features  being  present  in  even  the  simplest 
fire  distribution  problem,  the  reader  might  well  ask,  "Can  anything 
be  done  to  circumvent  such  mathematical  complexities?"  We  believe 
that  this  can  be  partially  done  by  identifying  the  circumstances  under 
which  such  difficult  mathematical  points  arise  in  the  model's  solution. 
In  other  words,  it  is  very  important  to  have  complete  (and  mathemati- 
cally correct)  solutions  to  the  simplest  tactical  allocation  problems 
if  for  no  other  reason  than  to  identify  which  formulations  lead  to 
prohibitive  mathematical  complexities  (and  hence  should  be  avoided) . 
For  example,  we  know  that  for  n-versus-one  combat  such  difficulties 
don't  arise  when  surviving  target  types  are  valued  in  direct  proportion 
to  their  kill  capability  as  measured  by  their  Lanchester  attrition- 
rate  coefficient. 

Finally,  except  for  the  references  cited  above  (and  pp.  276-284 
of  [4]),  we  could  find  no  other  treatment  of  multiple  extremals  in  the 
current  literature  of  deterministic  optimal  control  theory. 
(2)   State  Variable  Inequality  Constraints. 

As  we  saw  in  our  discussions  in  Section  1  above,  all  tactical 
allocation  problems  contain  negativity  restrictions  on  the  force 
levels  due  to  physical  reasons  (negative  force  levels  don't  make  sense). 
In  general,  force  levels  will  be  represented  by  state  variables  so  that 
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such  a  restriction  is  mathematically  called  a  state  variable  inequality 
constraint  (frequently  denoted  as  SVIC) .   The  maximum  principle  (in 
its  original  form  [101])  is  inadequate  to  handle  SVIC's  and  special 
mathematical  theories  have  been  developed  for  state-variable-inequality- 
constrained  problems.   When  we  started  our  research,  we  found  that 
results  were  widely  scattered  in  the  literature  and  that  a  completely 
adequate  theory  to  solve  even  the  simplest  fire  distribution  problem  did 
not  exist.   Hence,  we  have  developed  a  theory  of  SVIC's  for  tactical 
allocation  problems  within  the  Lanchester  theory  of  combat  (see 
Appendix  E) . 

A  mathematical  theory  of  SVIC's  is  essential  for  solving  tac- 
tical allocation  problems  within  the  Lanchester  theory  of  combat. 
Using  such  a  theory,  one  can  determine  when  it  is  not  optimal  to 
drive  a  force  level  to  zero,  the  optimal  order  in  which  to  annihilate 
target  types,  etc.   Thus,  the  mathematical  theory  (necessary  conditions) 
allows  one  to  (significantly)  reduce  the  volume  of  allocation  strategy 
combinations  which  one  has  to  examine  in  solving  a  differential  game 
(thus  circumventing  the  computational  difficulty  encountered  by 
Sternberg  (see  pp.  154-155  of  [111])). 

As  noted  frequently  in  this  report,  no  adequate  theory  of  SVIC's 
exists  within  the  current  mathematical  theory  of  differential  games. 


*  t 

A.  Friedman,  of  course,  has  treated  differential  games  with  SVIC  s 

(i.e.  games  with  restricted  phase  coordinates) [52] .   His  approach  is 
different  than  the  variational  approach  used  in  optimal  control  theory, 
and  his  emphasis  is  on  existence  theorems.   Moreover,  his  treatment 
(see  pp.  239-240  of  [52])  of  Isaacs'  war  of  attrition  and  attack  (see 
pp.  96-104  of  [72])  implicitly  requires  an  unsupported  assumption. 
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The  development  of  a  mathematical  theory  of  SVIC's  for  differential 
games  which  occur  within  the  Lanchester  theory  of  combat  appears  to 
be  the  most  needed  extension  of  the  existing  theory  of  differential 
games  for  military  operations  research  applications. 

Although  optimal  control  problems  involving  inequality  con- 
straints on  a  function  of  the  state  variables  with  no  explicit 
dependence  upon  the  control  variables  have  received  fairly  compre- 
hensive treatment  in  recent  years  (see  [77],  [79],  [94],  [115]  for 
comprehensive  bibliographies) ,  most  results  are  widely  scattered  in 
the  literature.   (Moreover,  the  theory  which  has  appeared  in  textbooks 
has  some  important  gaps  in  it  and  is  inadequate  (by  itself)  to  solve 
the  fire  distribution  problems  that  have  been  the  subject  of  our 
research  [115].)  The  poineering  work  of  the  Russian  R.  Gamkrelidze  in 
1959  (see  Chapter  VI  in  [101])  was  followed  in  the  USA  by  that  of 
L.  Berkovitz  [9],  S.  Dreyfus  [47],  and  Bryson,  Denham,  and  Dreyfus 
[29].   Berkovitz  and  Dreyfus  [15]  subsequently  have  shown  the  equiva- 
lence of  the  results  of  Gamkrelidze  and  Berkovitz  with  those  of 
Dreyfus. 

Mclntyre  and  Paiewonsky  [94]  have  written  an  excellent  survey 
article  on  the  theory  of  SVIC's  and  summarize  theoretical  results 
through  1966.   Subsequent  work  has  been  by  Jacobson  and  Lele  [77]  and 
Jacobson,  Lele,  and  Speyer  [79],  the  latter  authors  deriving  their 
results  by  considering  generalized  Kuhn-Tucker  conditions  in  a  Banach 
space.   Sufficient  conditions  of  optimality  for  problems  with  SVIC's 
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have  been  given  by  Funk  and  Gilbert  [53].   Finally,  we  would  like 
to  mention  the  work  by  two  students  of  M.  Hestenes  [55],  [102]. 

Moreover,  as  we  have  recently  pointed  out  [115],  practitioners 
of  optimal  control  have  not  always  been  aware  of  the  precise  content 
of  the  above  work.   We  are  aware  of  no  previous  work  relating  the 
theory  of  SVIC's  to  allocation  problems  in  the  Lanchester  theory  of 
combat . 

(3)   Singular  Subarcs. 

As  we  have  seen  in  Section  1  above,  tactical  allocation  problems 
within  the  Lanchester  theory  of  combat  may  be  formulated  (see  equations 
(1)  above)  as  differential  games  in  which  the  strategy  variables  (e.g. 
fraction  of  X.   who  fire  at  Y.)   appear  linearly.   Such  problems 
are  called  singular  (because  the  matrix  of  second  partial  derivatives 
of  the  Hamiltonian  with  respect  to  the  strategy  variables  is  a  singular 
matrix)  and  require  special  treatment.   One-sided  versions  of  such 
fire  distribution  problems  then  are  singluar  problems  of  optimal 
control.   A  discussion  of  terminology  and  the  mathematical  aspects  of 
singular  optimal  control  problems  has  been  given  in  Section  l.b.  above. 
We  saw  there  that  the  maximum  principle  does  not  determine  the  singular 
control  and  that  the  classical  theory  does  not  provide  adequate  tests 
of  optimality  for  singular  subarcs. 

A  mathematical  theory  of  singular  subarcs  is  essential  for 
solving  tactical  allocation  problems  within  the  Lanchester  theory  of 
combat  because  it  is  required  to  determine  the  optimality  of  fire 
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distribution  policies  that  are  not  extreme  points  of  the  control 
variable  space  (i.e.  policies  that  are  other  than  0  or   1).   We 
have  seen  that  an  optimal  fire  distribution  policy,  denoted  as  <f>  , 
such  that   0  <  <J>   <  1   arises  in  the  simple  problem  of  target  selec- 
tion for  a  homogeneous  force  in  Lanchester  combat  against  two  enemy 
force  types  each  of  which  undergoes  attrition  at  a  rate  proportional 
to  the  product  of  the  numbers  of  f irers  and  targets  [120] . 

The  establishment  of  the  singular  subarcs  in  such  a  problem  as 
above  requires  examination  of  the  generalized  Legendre-Clebsch  condi- 
tion (a  second  order  condition) .   In  a  problem  such  as  (11)  in  which 
target-type  attrition  corresponds  to  Lanchester 's  classical  square-law 
attrition  process  (i.e.  the  attrition  rate  of  a  target  type  is  propor- 
tional only  to  the  number  of  firers)  the  absence  of  singular  subarcs 
is  established  by  showing  that  it  is  impossible  to  have  (except  for 

pathological  cases)   —  f— — j  =  0,   where  H  denotes  the  Hamiltonian. 

at  vd<py 

The  importance  of  the  generalized  Legendre-Clebsch  condition  is  empha- 
sized by  the  fact  that  we  have  encountered  a  problem  (suggested  by 
the  work  of  P.  Chaiken)  in  which  it  is  the  only  way  to  establish  the 
non-optimality  of  splitting  forces  in  combat. 

Singular  problems  of  optimal  control  were  apparently  first 
explicitly  studied  in  1963  by  Johnson  and  Gibson  [81]  (see  [80]  for 
a  discussion  of  singular  solutions  in  optimal  control  problems  within 
a  broad  perspective  of  applied  mathematics) .   Since  that  time  the 
subject  has  grown  to  comparative  maturity  although  most  recent  results 


44 


are  widely  scattered  in  the  literature.   Surveys  of  early  work  are 
contained  in  the  works  by  Kelley,  Kopp,  and  Moyer  [83]  (summarizes 
results  through  1966)  and  Bryson  and  Ho  (see  Chapter  8  of  [28]) 
(summarizes  results  through  1968) .   The  above  work  is  sufficient  to 
adequately  treat  all  the  singular  problems  of  the  optimal  control  of 
deterministic  Lanchester  processes  which  we  have  so  far  encountered. 

Kelly  [82]  ,  [83]  apparently  first  generalized  the  Legendre- 
Clebsch  necessary  conditions  to  singular  subarcs.   This  was  further 
generalized  to  singular  extremals  involving  several  control  variables 
by  Goh  [54],   Jacobson  later  derived  an  additional  necessary  condition 
via  differential  dynamic  programming  [75].   Very  recently,  Jacobson 
[76]  has  given  a  general  sufficiency  theorem  for  the  second  variation 
which  allows  one  to  treat  both  singular  and  non-singular  problems. 
This  yields  conditions  analogous  to  the  well-known  no-conjugate- 
point  condition  (see  pp.  181-184  of  [28])  for  singular  subarcs  and  has 
led  to  necessary  and  sufficient  conditions  of  optimality  for  singular 
control  problems  [110]  (see  also  [78]). 

Except  for  our  own  work  [121],  we  are  aware  of  no  previous 
work  applying  the  theory  of  singular  extremals  to  tactical  allocation 
problems  within  the  Lanchester  theory  of  combat. 

d.   Optimal  Control  of  the  Lanchester  Stochastic  Process. 

A  totally  underdeveloped  area  in  the  Lanchester  theory  of  combat 
has  been  optimal  target  section  in  combat  among  heterogeneous  forces 
when  attrition  is  a  stochastic  process.   Based  on  our  review  of  the 
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pertinent  literature  (see  Section  1  of  Appendix  I) ,  we  conclude  that 
there  has  been  no  past  adequate  treatment  of  stochastic  versions  of 
the  fire  distribution  problems  that  we  have  previously  studied 
using  deterministic  models.   By  studying  such  stochastic  versions, 
we  plan  to  see  if  the  "best"  tactics  for  target  selection  are  depend- 
ent upon  how  attrition  is  modelled,  i.e.  deterministic  or  stochastic 
process.   In  this  report  we  present  preliminary  results  (development 
of  the  fundamental  functional  equation  for  the  optimal  expected  value 
function,  its  analytic  solution  in  some  special  cases,  and  a  numerical 
approximation  for  the  general  case) .   Although  our  results  are  special 
cases  of  well-known  results  in  the  field  of  optimal  stochastic  control, 
the  optimal  control  of  the  Lanchester  stochastic  process  had  apparently 
never  been  studied  in  the  past  by  this  approach. 

In  Appendix  I  we  consider  a  battle  which  is  to  last  a  prescribed 
length  of  time  as  an  optimal  stochastic  control  problem.   Our  idea  is 
to  compare  the  structure  of  the  optimal  allocation  policies  between 
deterministic  and  stochastic  formulations.   Accordingly,  we  have 
developed  a  complete  "solution"  to  the  corresponding  deterministic 
control  problem  (see  Appendix  G) .   We  are  currently  directing  a 
graduate  student  (LT  R.  Powers,  USN)  in  making  a  numerical  comparison 
between  results  obtained  by  these  two  models,  and  we  will  report 
these  results  at  a  later  date. 
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5.   Guided  Tour  of  the  Appendices. 

In  this  section  we  summarize  the  work  which  is  contained  in 
the  appendices  and  explain  why  this  work  was  done.   In  our  first  report 
[113]  we  emphasized  the  extension  of  results  which  had  previously 
appeared  in  the  literature.   In  the  present  report  we  give  further 
extensions  and,  moreover,  develop  the  mathematical  foundations  for 
more  extensive  future  work. 

In  Appendix  A  we  derive  a  complete  solution  to  the  Isbell  and 
Marlow  fire  programming  problem  [74].   This  current  work  revises 
earlier  preliminary  results  on  this  problem.   We  develop  general 
methodology  for  solving  such  deterministic  optimal  control  problems 
and  then  demonstrate  its  application  by  developing  a  complete  solution 
(however,  further  extensions  are  given  in  Appendix  F)  for  a  terminal 
control  battle  (the  battle  only  terminates  when  the  course  of  battle 
has  reached  some  specified  state)  between  a  homogeneous  force  and  a 
heterogeneous  enemy  force  of  two  target  types.   Our  methodology  is 
more  general,  though,  and  may  be  applied  to  the  cases  of  one-versus-n 
target  types,  variable  attrition-rate  coefficients,  etc. 

The  structure  of  the  optimal  fire  distribution  policy  for  the 
Isbell-Marlow  problem  is  discussed.   We  show  how  the  optimal  fire 
distribution  policy  may  depend  indirectly  upon  the  force  levels.   For 
the  attrition  model  used  (attrition  rate  of  target  types  proportional 
to  the  number  of  firers  only)  the  optimal  policy  is  always  to  concen- 
trate all  fire  on  one  target  type.   Our  work  extends  that  of  Isbell 
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and  Marlow  [74]  by  showing  that  the  solution  to  this  optimal  control 
problem  contains  dispersal  furfaces.   Additionally,  we  show  how  to 
determine  the  domain  of  controllability  for  extremals  to  each  terminal 
state. 

The  work  presented  in  Appendix  A  was  done  because  the  Isbell- 
Marlow  problem  [74]  represents  the  simplest  terminal  control  fire 
distribution  problem.   Any  mathematical  theory  developed  to  handle 
more  complex  tactical  allocation  problems  of  greater  practical  signi- 
ficance must  be  able  to  adequately  handle  this  simple  case. 

In  Appendix  B  we  examine  the  structure  of  optimal  allocation 
policies  for  tactical  situations  described  by  Lanchester-type  models 
of  warfare  by  studying  a  sequence  of  simplified  models.   We  do  this 
because  no  previous  work  had  ever  systematically  investigated  the 
dependence  of  tactics  upon  model  form.   We  consider  models  for  two 
types  of  choice  situations 

(1)  selection  of  target  type, 

(2)  regulation  of  firing  rate. 

These  problems  are  solved  by  the  mathematical  theory  of  optimal 
control. 

In  the  first  sequence  of  models  we  examine  the  effects  on  the 
optimal  target  selection  policy  of  the  following  factors:   objectives 
of  the  combatants,  termination  conditions  of  the  conflict,  number  of 
target  types,  some  special  cases  of  time-dependent  attrition-rate 
coefficients,  and  type  of  attrition  process.   We  then  examine  a  sequence 
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of  models  to  see  how  ammunition  limitations  affect  firing  rates.   Next 
we  discuss  two-sided  extensions  of  such  problems  but  point  out  the 
value  of  studying  one-sided  problems  as  considered  in  this  report. 
Finally,  various  implications  of  the  models  are  discussed.   Some  of 
the  results  presented  in  this  appendix  are  further  extended  in 
Appendix  G. 

In  Appendix  C  we  develop  partial  solutions  to  a  sequence  of 
models  for  the  optimal  fire  distribution  policy  for  a  homogeneous 
force  in  combat  against  geterogeneous  enemy  forces.   We  consider  a 
prescribed  duration  battle  and  concentrate  on  the  special  case  when 
no  force  type  is  annihilated  during  combat.   For  both  one-versus-two- 
target-types  combat  and  one-versus-n-target-types  combat  we  develop 
our  results  for  both  constant  attrition-rate  coefficients  and  also 
some  special  cases  of  variable  attrition-coefficients. 

We  undertook  the  work  of  Appendix  C  to  show  how  our  results 
for  one-versus-two-target-types  combat  could  be  extended  to  one- 
versus-n  combat  and  variable  attrition-rate  coefficients  (thus  closer 
modelling  real  world  complexities).   In  this  work  we  draw  heavily 
upon  our  research  on  homogeneous  force  models  with  variable 
attrition-rate  coefficients  [114],  [116],  [123]  (see  also  Appendix 
H) .   First,  we  consider  the  problem  of  optimal  fire  distribution  over 
two  target  types  for  some  special  cases  of  variable  attrition-rate 
coefficients.   Next,  we  consider  several  target  types  and  constant 
attrition-rate  coefficients.   Then,  we  treat  the  more  general  case 
of  several  target  types  for  some  special  instances  of  variable 
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attrition-rate  coefficients.   Finally,  we  make  some  observations  on 
the  results  presented  in  this  appendix. 

Our  results  presented  in  Appendix  C  show  that  some  special 
cases  of  variable  attrition-rate  coefficients  are  mathematically 
tractable  for  optimal  control  problems  within  the  Lanchester  theory 
of  combat.   Additionally,  we  show  that  in  one-versus-n   combat  the 
solution  to  the  optimal  fire  distribution  problem  takes  a  particularly 
simple  form  when  enemy  target  types  are  valued  (linearly)  in  direct 
proportion  to  their  kill  capability  (as  measured  by  the  Lanchester 
attrition-rate  coefficient) .   In  this  special  case,  the  optimal  fire 
distribution  policy  is  particularly  simple:   namely,  concentrate  all 
fire  upon  the  available  target  type  with  the  largest  product  of 
attrition-rate  coefficients  (i.e.  largest  a.b.).   This  is  even  true 
for  some  special  cases  of  variable  attrition-rate  coefficients. 

In  Appendix  D  we  treat  a  simple  problem  of  fire  distribution 
for  a  homogeneous  force  in  Lanchester  combat  against  two  enemy  force 
types,  each  of  which  undergoes  attrition  at  a  rate  proportional  to  the 
product  of  the  numbers  of  firers  and  targets.   We  undertook  this  work 
in  order  to  study  the  dependence  of  optimal  tactics  upon  the  attri- 
tion model.   In  addition  to  the  Pontryagin  maximum  principle,  the 
theory  of  singular  extremals  is  required  to  solve  this  problem. 

Our  major  result  is  that  for  fire  distribution  problems  in 
which  target-type  attrition  corresponds  to  the  classical  Lanchester 
linear-law  attrition  model  "the  optimal  fire  distribution  policy  is 
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not  always  to  concentrate  all  fire  on  a  single  target  type.   There  are 
circumstances  when  the  optimal  policy  is  to  split  one's  fire  between 
the  two  target  types.   Additionally,  we  show  how  to  synthesize  the 
optimal  fire  distribution  policy  from  the  basic  optimality  conditions. 
For  constant  attrition-rate  coefficients  we  show  that  whether  or  not 
changes  can  occur  in  target  priorities  depends  solely  on  how  survivors 
are  valued  and  is  independent  of  the  type  of  attrition  process. 

In  Appendix  E  we  summarize  results  for  the  theory  of  state 
variable  inequality  constraints  (frequently  denoted  as  SVIC's)  which 
had  previously  been  widely  scattered  in  the  literature.   Then  we  apply 
this  theory  to  two  tactical  allocation  problems.   The  first  problem 
(tactical  air-war  campaign)  is  a  one-sided  version  of  R.  Isaacs' 
"War  of  Attrition  and  Attack"  [72].   However,  the  non-negativity  of 
the  force  levels  was  not  adequately  handled  mathematically  in  past 
work  [72],  and  we  consider  versions  of  the  problem  for  which  the 
previous  method  of  analysis  does  not  lead  to  an  optimal  policy.   The 
second  problem  is  the  fire  programming  problem  studied  by  Isbell  and 
Mar low  [74].   We  consider  several  versions  of  both  these  problems. 
For  one-versus-n-target-types  combat  we  extend  our  past  results  by 
developing  necessary  conditions  of  optimality  for  annihilating  a 
target  type  in  two  important  cases:   (1)  two  target  types  with  a 
special  case  of  variable  attrition-rate  coefficients  and  (2)   n 
target  types  with  constant  attrition-rate  coefficients. 
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We  undertook  the  work  presented  in  Appendix  E  since  no  past 
work  on  optimal  tactical  allocation  had  given  an  adequate  treatment 
of  the  non-negativity  restrictions  on  force  levels  in  developing 
optimal  policies.   In  our  own  previous  work  on  the  Isbell-Marlow 
problem  [74]  we  followed  the  heuristic  treatment  originated  by  Isbell 
and  Marlow  [74]  (who,  in  turn,  were  following  Isaacs  [70])  for  deter- 
mining the  boundary  conditions  for  the  dual  variables.   Our  work  here 
establishes  a  firm  theoretical  foundation  for  such  important  aspects 
and  justifies  certain  assumptions  that  we  made  in  developing  our 
solution  to  the  Isbell-Marlow  problem  [117].   Thus,  we  re-consider 
the  development  of  certain  key  necessary  conditions  of  optimality 
within  the  framework  of  the  theory  of  SVIC's. 

In  Appendix  F  we  consider  some  further  aspects  of  the  solution 
to  the  Isbell  and  Marlow  fire  programming  problem.   We  show  how  the 
Isbell-Marlow  problem  is  solved  using  the  theory  of  SVIC's.   Addition- 
ally, we  show  that  for  certain  values  of  model  parameters  (1)  the 
optimal  policy  may  not  be  unique  and  (2)  in  treating  multiple  extremals 
there  is  a  dominated  return  (i.e.  the  return  associated  with  one  family 
of  extremals  dominates  that  associated  with  another  family  of  extremals) 
besides  the  dispersal  surfaces  in  this  problem's  solution.   We  under- 
took this  work  because  our  treatment  of  more  complex  tactical  allocation 
problems  relies  heavily  on  the  complete  solution  of  simpler  problems. 
This  important  theory  (SVIC's)  had  never  been  applied  previously  to 
determine  the  optimal  control  of  deterministic  Lanchester  processes. 
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In  Appendix  G  we  develop  a  complete  solution  to  the  prescribed 
duration  battle  using  the  theory  of  SVIC's.   We  do  this  in  order  to 
compare  the  structure  of  the  optimal  fire  distribution  policy  for  the 
prescribed  duration  battle  problem  with  that  of  other  problems  (i.e. 
terminal  control  battle  and  optimal  control  of  the  Lanchester  stochas- 
tic process) .   We  give  a  general  solution  algorithm  that  applies  to 
all  such  deterministic  optimal  control  problems.   We  have  used  this 
algorithm  to  generate  all  the  results  given  in  this  report.   Discus- 
sions of  the  following  important  topics  are  given  :   (1)   the  structure 
of  the  optimal  fire  distribution  policy  and  (2)  the  development  of 
solutions  to  such  problems.   We  show  that  even  when  target  types 
undergo  a  "square-law"  attrition  process,  the  optimal  allocation 
policy  may  depend  indirectly  upon  the  force  levels. 

In  Appendix  H  we  develop  solutions  to  extensions  of  F.  W. 
Lanchester' s  classical  equations  of  modern  warfare  (frequently  referred 
to  as  aimed-fire  equations)  for  combat  between  two  homogeneous  forces. 
In  these  extensions  the  lethality  of  the  fire  (as  expressed  by  the 
Lanchester  attrition-rate  coefficient)  depends  upon  time.   When  the 
dependence  is  arbitrary,  the  solution  is  an  infinite  series  of  recur- 
sively related  integrals;  in  special  cases,  more  convenient  representa- 
tions (including  representation  in  terms  of  tabulated  functions)  are 
available.   Solutions  are  obtained  in  the  following  cases:   (1) 
lethality  of  each  side's  fire  proportional  to  a  power  of  time  and  both 
lethalities  initially  zero,  and  (2)  lethality  of  each  side's  fire 
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linear  with  time  but  only  one  side's  lethality  initially  zero.  The 
latter  case  models  the  constant  speed  approach  between  forces  whose 
weapons  have  different  maximum  effective  ranges.  We  undertook  this 
work  because  we  could  find  no  adequate  treatment  available  in  the 
literature.  These  results  have  been  subsequently  used  for  problems 
on  the  optimal  control  of  deterministic  Lanchester  processes  with 
variable  attrition-rate  coefficients. 

In  Appendix  I  we  present  preliminary  results  on  the  optimal 
control  of  the  Lanchester  stochastic  process.   We  consider  a  stochastic 
version  of  the  prescribed  duration  fire  distribution  problem  studied 
in  Appendix  G.   Our  purpose  for  doing  this  work  is  to  determine 
whether  the  structure  of  the  optimal  allocation  policy  is  affected  by 
whether  the  attrition  process  is  modelled  as  being  deterministic  or 
stochastic. 

Thus,  in  Appendix  I  we  develop  the  fundamental  functional 
equation  satisfied  by  the  optimal  expected  value  function  and  derive 
solutions  in  some  special  cases.   Except  for  these  special  cases, 
the  fundamental  functional  equation  (actually  a  system  of  differential- 
difference  equations)  possesses  a  solution  that  is  too  complex  for  the 
development  of  a  closed-form  analytic  solution.   Therefore  we  have 
developed  a  finite  difference  approximation  which  can  be  used  to 
generate  numerical  solutions  for  small  numbers  of  combatants. 

Using  the  above  finite  difference  approximation,  a  thesis  student 
has  done  some  numerical  computations  for  us.   It  has  only  proven  to 
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be  feasible  to  use  the  computer  to  generate  numerical  results  for 
battles  with  up  to  10  Y's  versus  10  X  's   and  10   X  's   due  to 
computer  memory  requirements.   Additionally,  we  have  considered 
approximations  valid  for  large  numbers  of  combatants.   In  pursuing 
this  approach,  we  have  developed  a  diffusion  approximation  to  the 
Lanchester  stochastic  process. 

In  the  fire  distribution  problems  studied  above  for  a  homog- 
enous force  in  combat  with  a  heterogeneous  enemy  it  has  always  been 
assumed  that  fire  can  always  be  instantaneously  re-distributed  from 
being  entirely  concentrated  on  one  target  type  to  any  other  one. 
For  example,  in  "two-on-one"  combat  the  decision  (or  control)  variable 
represents  the  fraction  of  Y   fire  that  is  directed  against  X  . 
Instantaneous  jumps  in  this  control  variable  (for  example,  from  0 
to   1)  have  been  permitted.   This  corresponds,  in  a  sense,  to  a 
"perfect"  command  and  control  capability.   In  Appendix  J  we  present 
preliminary  results  for  the  more  realistic  assumption  that  there  is 
a  limit  to  how  fast   <J)   can  be  changed  corresponding  to  command  and 
control  limitations. 

We  undertook  this  work  presented  in  Appendix  J  in  order  to 
explore  the  effects  of  command  and  control  capabilities  upon  the 
structure  of  the  optimal  fire  distribution  policies.   We  show  that 
the  presence  of  command  and  control  limitations  (which  cause  the  rate 
of  re-distribution  of  fire  to  be  bounded)  causes  fire  to  be  shifted 
earlier  than  when  fires  can  be  instantaneously  re-distributed  in 
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anticipation  of  changes  in  target  priorities.   This  phenomenon  is 
analogous  to  the  situation  when  with  reduced  reaction  times,  one 
tries  to  "anticipate"  the  actions  of  an  enemy.   The  development  of 
these  results  has  required  some  advanced  theory  for  SVIC's  (treatment 
of  a  second  order  SVIC) . 

Finally,  in  Appendix  K  we  further  investigate  the  sensitivity 
of  the  optimal  target  selection  policy  to  the  form  of  the  combat 
attrition  model  by  considering  a  two-against-one-target-type  selec- 
tion problem  with  combat  attrition  following  a  general  form  proposed 
by  R.  Helmbold.   For  combat  between  homogeneous  forces  Helmbold  [59] 
has  proposed  a  general  Lanchester-type  model  in  which  the  effective- 
ness of  a  force  is  dependent  upon  the  force  ratio.   This  general 
formulation  yields  Lanchester's  square  law  and  Lanchester's  linear 
law  as  special  cases.   We  extend  Helmbold' s  model  to  combat  between 
heterogeneous  forces  and  use  this  extension  as  the  attrition  model 
in  a  tactical  allocation  optimization  problem. 

In  Appendix  K,  then,  we  present  preliminary  results  for  optimal 
target  selection  for  Helmbold' s  general  attrition  structure.   These 
preliminary  results  indicate  that  when  target-type  attrition  follows 
Helmbold 's  general  model,  the  optimal  policy  is  always  to  concentrate 
all  fire  on  a  single  target  type.   (In  other  words,  we  could  find  no 
singular  solution  in  this  case.)   This  should  be  contrasted  with  our 
result  [121]  that  when  target  types  undergo  attrition  corresponding  to 
a  "linear-law"  Lanchester  process,  it  is  sometimes  optimal  to  split 
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one's  fire.   In  other  words,  as  far  as  the  optimal  distribution  of  fire 
is  concerned,  Helmbold's  general  attrition  structure  behaves  like 
that  in  which  a  target  type's  attrition  rate  is  proportional  only  to 
the  number  of  firers. 

6 .   Relevance  to  Current  Navy  Problems. 

In  this  section  we  explain  the  relevance  of  our  research  on 
the  optimal  control  of  Lanchester  processes  to  current  Navy  problems. 

a.   Our  theoretical  work  on  tactical  allocation  problems  is 
being  interfaced  with  the  ONR  supported  study  effort  of  P.  Chaiken  of 
Stanford  Research  Institute.   Chaiken  has  pioneered  [31],  [32],  [33], 
[34]  in  the  application  of  quantitative  (Lanchester-type)  methodology 
for  force  level  planning,  development  of  contingency  plans,  etc.   This 
methodology  has  been  used  for  significant  force  level  problems  defined 
by  the  military — Commander-in-Chief -Pacific  (CINPAC) ,  Headquarters 
Marine  Corps  (HQMC) ,  and  Chief  of  Naval  Operations  (CNO)  [34]. 

To  determine  optimal  allocation  strategies  in  dynamic  problems, 
Chaiken  uses  a  heuristic  optimization  approach  of  assuming  a  certain 
type  of  strategy  (constant  or  piecewise  constant)  and  then  comparing 
computed  campaign  outcomes  for  various  levels  of  strategy  variables 
(essentially,  a  so-called  direct  method  of  optimization  [90]).   Our 
research  supports  his  study  objectives  by  (1)  justifying  such  assump- 
tions, (2)  suggesting  strategy  alternatives  to  explore  in  more  detailed 
computer  simulations  such  as  BALFRAM  [37],  (3)  explicitly  showing  the 
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effects  of  modelling  assumptions  on  optimal  tactical  allocation 
strategies,  (4)  developing  insights  into  optimal  allocation  strate- 
gies for  various  attrition  structures  of  interest. 

b.  Thus,  we  are  striving  to  provide  a  sound  theoretical  basis 
for  application  efforts  such  as  by  Chaiken.   In  our  present  work  we 
have  concentrated  on  solution  techniques  and  methodology,  since  this 
was  a  gap  in  the  current  state-of-the-art  for  applications  of  optimal 
control/differential  game  theory  to  Lanchester-type  models  of  warfare. 
We  have  studied  idealizations  (abstractions)  of  allocation  structures 
arising  in  more  complex  (realistic)  defense  planning  studies.   Through 
such  idealization  and/or  simplification  we  have  tried  to  identify 
factors  or  concepts  which  merit  further  investigation  in  more  detailed 
computer  simulations  (such  as  BALFRAM  [37]).   For  example,  uses  have 
never  exercised  the  "linear-law"  attrition  option  of  MILISTRAC  and 
BALFRAM  [36].   Yet  our  research  indicates  that  optimal  tactics  with 
"linear-law"  attrition  are  fundamentally  different  than  those  with 
"square-law"  attrition.   Hence,  planners  should  be  alerted  (educated) 
to  this  fact  about  the  dependence  of  optimal  tactics  on  type  of 
attrition  process. 

c.  It  seems  appropriate  to  briefly  discuss  a  modelling  issue 
of  abstraction  versus  realism.   The  mass  of  details  present  in  a 
realistic  (complex)  model  inevitably  obscures  essential  modelling 
issues  such  as  type  of  attrition  process,  type  of  objective  (criterion) 
function.   Our  work  is  oriented  towards  clarifying  basic  issues  by 
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examining  simplified  models.   For  example,  versions  of  a  well-known 
model  (see  pp.  96-104  in  [72])  which  are  used  for  justifying  tactics 
in  the  air  war  consider  a  combat  payoff  in  terms  of  net  sorties  of 
ground  support  missions  flown  and  totally  ignore  the  effects  of  such 
sorties  on  the  outcome  of  the  ground  war.   Our  theoretical  analysis 
of  this  situation  provides  motivation  (and  justification)  for  a 
current  ONR  study  [35]  to  determine  the  optimal  tactics  of  the  air 
war  in  the  context  of  the  ground  war  objectives  for  contingency  plans 
of  interest. 

d.  A  sequential  game  arises  in  a  current  application  of  P. 
Chaiken's  BALFRAM  methodology  [35].   Our  research  lays  the  theoretical 
foundations  (via  the  research  on  differential  games)  for  the  inclusion 
of  the  capability  to  handle  such  a  sequential  game  within  the  BALFRAM 
software  system. 

e.  The  questions  of  target  priority  rules  for  target  selection 
frequently  arises  in  the  evaluation  of  proposed  weapon  systems  and/or 
doctrine  of  employment  for  weapon  systems.   The  study  of  idealized 
combat  models  would  provide  insight  into  the  optimization  of  combat 
dynamics  hopefully  leading  to  a  better  quantitative  basis  for  defense 
planning  decisions. 

f.  In  the  Navy  mission  of  fire  support,  the  question  of  target 
priority  rules  for  target  selection  frequently  arises.   An  understanding 
of  optimization  principles  in  various  dynamic  combat  situations  would 
lead  to  better  utilization  of  such  resources. 
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7 .   Summary  of  Research  Findings. 

Here  we  summarize  our  research  results.   We  found  that  solution 
techniques  were  insufficiently  developed  for  the  optimization  of 
combat  dynamics.   Accordingly,  this  aspect  received  top  priority  in 
our  work.   Results  are  organized  under  the  following  headings: 

(1)  solution  techniques, 

(2)  insights  into  optimal  tactical  allocation, 

(3)  implications  for  defense  planning. 

Items  (2)  and  (3)  differ  in  that  the  latter  is  a  management-oriented 
digest  of  the  practical  implications  of  our  research  whereas  the 
former  is  oriented  towards  a  technical  audience.   Further  amplification 
of  results  and  conclusions  is  to  be  found  in  the  appendices. 

a.   Solution  Techniques. 

Our  research  has  produced  the  following  results  on  solution 
techniques  for  the  optimization  of  combat  dynamics.   Specifically,  we 
have  accomplished  the  following: 

(1)  developed  general  solution  algorithm  (based  on  results  from 
modern  optimal  control  theory)  to  determine  the  optimal 
control  of  deterministic  Lanchester  processes, 

(2)  developed  the  basic  equations  of  optimality  for  the  optimal 
control  of  the  Lanchester  stochastic  process, 

(3)  developed  a  theory  (theory  of  state  variable  inequality 
constraints)  for  the  adequate  treatment  of  non-negativity 
restrictions  on  force  levels  in  optimal  control  problems 
for  deterministic  Lanchester  processes, 

(4)  discovered  that  no  adequate  theory  of  state  variable  inequality 
contraints  exists  in  current  theories  for  differential  games, 
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(5)  shown  how  to  treat  the  following  difficult  aspects  of  optimal 
control  problems  for  deterministic  Lanchester  processes: 

(a)  singular  solutions, 

(b)  state  variable  inequality  constraints, 

(c)  multiple  extremals  (including  dispersal  surfaces  and 
dominated  returns) , 

(6)  developed  complete  solutions  for  optimal  fire  distribution 
tactics  in  one-against-two-target-types  Lanchester  combat  for 
two  cases : 

(a)  terminal  control  battle  (f ight-to-the-f inish) , 

(b)  prescribed  duration  battle, 

(7)  developed  partial  solutions  for  optimal  fire  distribution  tactics 
for  a  force  engaged  in  Lanchester  combat  for  the  following 
cases : 

(a)  one-against-two-target-types  combat, 

(A)  some  special  cases  of  variable  attrition-rate 
coefficients, 

(B)  "linear-law"  attrition  of  target  types, 

(C)  bounded  rates  for  changing  the  distribution  of  fire, 

(D)  Helmbold's  general  attrition  model, 

(E)  stochastic  attrition  process, 

(b)  one-against-n-target-types  combat, 

(A)  constant  attrition-rate  coefficients, 

(B)  some  special  cases  of  variable  attrition-rate 
coefficients, 

(8)  developed  solutions  to  homogeneous  force  models  with  variable 
attrition-rate  coefficients. 


* 


In  (6)  and  (7)  the  attrition  process  is  deterministic  and  the  attrition 
rate  of  a  target  type  is  proportional  to  only  the  number  of  firers 
unless  otherwise  noted. 
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b .   Insights  into  Optimal  Tactical  Allocation. 

Based  on  our  study  of  the  optimization  of  combat  dynamics  using 
the  mathematical  theories  of  differential  games/optimal  control  we 
have  reached  the  following  conclusions: 

(1)  the  structure  of  the  optimal  allocation  policy  for  fire 
distribution/target  selection  depends  upon  model  form;  more 
specifically,  the  optimal  policy  depends  on  force  levels, 
weapon  system  capabilities  (as  measured  by  Lanchester  attri- 
tion-rate coefficients),  the  type  of  attrition  process,  the 
target  acquisition  process,  the  values  placed  upon  surviving 
force  types,  and  the  termination  conditions  of  combat, 

(2)  for  deterministic  Lanchester  one-versus-n-target-types 
combat  in  which  the  attrition  rate  for  each  enemy  target  type 
is  proportional  to  only  the  number  of  firers,  the  optimal 
fire  distribution  policy  has  the  following  properties  (for 
linear  utility  of  survivors  and  no  replacements) : 

(a)  fire  is  always  concentrated  on  a  single  target  type 
(which  may  change  without  the  annihilation  of  the 
previous  target  type) , 

(b)  it  may  depend  indirectly  upon  force  levels  (depending 
upon  how  survivors  are  valued) , 

(c)  when  enemy  survivors  are  valued  in  direct  proportion  to 
their  kill  capability  (as  measured  by  their  Lanchester 
attrition-rate  coefficient  against  the  homogeneous  force) , 

(A)   for  constant  attrition-rate  coefficients 

(I)  fire  is  always  concentrated  on  the  available 
enemy  target  type  which  has  the  largest  product 
of  attrition-rate  coefficients  (i.e.  the  largest 
a.b.   where  a.   represents  the  kill  capability 
or  the  friendly  force  against  the  enemy's  ith 
target  type  and  b.   represents  that  of  the 
enemy's  ith  force  type), 

(II)  fire  is  only  shifted  to  a  new  target  type  after 
the  annihilation  of  the  previously  available 
target  type  with  the  largest  product  of  attri- 
rate  coefficients  (i.e.  no  change  in  fire  dis- 
tribution without  the  annihilation  of  a  target 

type) , 
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(B)   for  variable  attrition-rate  coefficients  in  the 

case  when  the  ratio  of  enemy  weapon  system  capabili- 
ties (as  measured  by  Lanchester  attrition-rate 
coefficients)  is  a  constant  for  any  two  enemy 
weapon  system  types 

(I)  assuming  that  no  target  type  is  annihilated, 
all  fire  is  always  concentrated  on  the  target 
type  with  the  largest  product  of  attrition- 
rate  coefficients, 

(II)  additional  assumptions  are  required  to  analyze 
the  case  when  a  target  type  is  annihilated, 

(d)   target  priorities  are  not  directly  influenced  by  replace- 
ment rates  when  these  are  added  to  the  model, 

(3)   for  deterministic  Lanchester  one-versus-two-target-types 

combat  in  which  the  attrition  rate  for  each  enemy  target  type 
is  proportional  to  the  product  of  the  numbers  of  firers  and 
target  types,  the  optimal  fire  distribution  policy  has  the 
following  properties  (for  constant  attrition-rate  coefficients, 
linear  utility  of  survivors,  and  no  replacements): 

(a)  it  depends  directly  upon  force  levels, 

w% 

(b)  (j>  (the  optimal  fraction  of  Y-fire  directed  at  X  ) 
takes  on  one  of  three  values:  0,1,  or  a_/(a  +a  J; 
thus,  it  is  sometimes  optimal  to  split  one's  fire, 

(c)  when  enemy  survivors  are  valued  in  direct  proportion  to 
their  kill  capability  (as  measured  by  their  attrition- 
rate  coefficients  against  the  homogeneous  force) 

(A)  the  ranking  of  target  priorities  is  never  reversed 
over  time, 

(B)  the  optimal  policy  is  given  explicitly  by 


* 


1      for  a  b  x  >  a  b  x  , 

a2 
;  — -  for  a^  =  a2b2x2, 

0      for   a  b  x  <  a2b2x2' 

(d)   the  structure  of  the  optimal  policy  is  not  sensitive  to 

the  type  of  attrition  sustained  by  the  homogeneous  force, 
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(4)  for  deterministic  Lanchester  one-versus-two-target-types 
combat  in  which  the  attrition  rate  for  each  target  type  is 
proportional  to  only  the  number  of  firers  with  (a)  constant 
attrition-rate  coefficients,  (b)  linear  utility  of  survivors, 
and  (c)  no  replacements,  the  optimal  fire  distribution  policy 
may  be  different  in  the  following  two  cases: 

(a)  prescribed  duration  battle, 

(b)  f ight-to-the-f inish, 

(This  difference  can  only  occur  when  for  a..b   >  a?b   enemy  target 
type  two  is  valued  in  greater  proportion  to  its  kill  capability  than 
target  type  one  is.) 

(5)  for  the  stochastic  version  of  the  prescribed  duration  battle 
described  in  (4)  above,  the  optimal  fire  distribution  policy 
sometimes  depends  upon  the  force  levels, 

(6)  for  the  battle  described  in  (4)  above,  when  there  is  an  upper 
bound  on  the  rate  of  changing  one's  distribution  of  fire 
(through  command  and  control  limitations) ,  the  structure  of 
the  optimal  fire  distribution  policy  is  not  appreciably 
changed;  the  following  statements  can  be  made  about  the 
optimal  policy: 

(a)  the  same  conditions  remain  for  it  to  be  optimal  to  have 
<j>*  =  0  or   1   (although  clearly  <j>*  may  have  to  take 
on  intermediate  values) , 

(b)  one  starts  to  shift  fire  earlier  (as  measured  by  the 
rate  of  destruction  of  target  type  value,  i.e.  product 
of  attrition-rate  coefficient  and  marginal  value  of 
target  type)  than  in  the  previous  case  when  $*     can 
instantaneously  jump,  for  example,  from  0   to   1, 

(7)  the  optimal  fire  distribution  policy  does  not  appear  to  be  the 
same  for  models  with  a  deterministic  attrition  process  and 
those  with  a  stochastic  attrition  process. 

c .   Implications  for  Defense  Planning. 

In  our  research  reported  here  we  have  studied  abstractions 

(idealizations)  of  allocation  structures  that  commonly  occur  in  defense 
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planning  studies.   After  studying  these  simplifications  in  order  to 
obtain  significant  information  about  the  structure  of  the  optimal 
allocation  strategies,  we  have  reached  the  following  conclusions 
(generalizations  of  phenomena  that  we  have  observed  in  simplified 
cases)  as  to  considerations  that  should  be  brought  to  the  attention 
of  defense  planners.   These  results  should  be  kept  in  mind  by 
practitioners  who  perform  more  detailed  computer  simulation  studies. 

(1)  The  nature  of  the  attrition  process  has  a  significant  effect 
upon  optimal  strategies.   At  present  we  feel  that  the  highest 
priority  should  be  given  to  educate  planners  to  the  fact  that 
the  type  of  optimal  allocation  strategy  is  dependent  on  the 
nature  of  the  attrition  model.   This  has  the  operational 
planning  implication  that  studies  should  be  run  with  both 
square-law  attrition  and  then  linear-law  attrition.   Past 
history  shows  that,  for  example,  the  linear-law  attrition 
option  has  never  been  exercised  in  BALFRAM  [36]. 

(2)  Force  levels  do  affect  optimal  strategies.   Whether  one  "wins' 
(superiority)  or  "loses"  (inferiority)  affects  optimal 
strategies . 

(3)  Even  the  nature  of  the  scenario  (terminal  control  n  pre- 
scribed duration  conflict)  may  affect  optimal  strategies. 
Thus,  if  one  develops  "good"  tactics  for  a  90  day  European 
campaign,  such  tactics  need  not  be  "good"  if  the  conflict 
does  not  terminate  at  the  prescribed  time. 

(4)  Optimal  tactics  for  the  air  war  are  dependent  upon  whether  or 
not  the  air  war  is  evaluated  in  the  context  of  ground  war 
objectives . 

(5)  Optimal  tactics  are  also  significantly  influenced  by  the 
nature  of  the  target  acquisition  process  and  command  and 
control  capabilities. 
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8.   Suggested  Future  Research  Tasks. 

After  performing  the  research  documented  in  this  report,  we 
feel  that  the  current  state-of-the-art  for  applying  optimal  control/ 
differential  game  theory  to  determine  optimal  allocation  strategies 
for  deterministic  Lanchester  processes  is  such  that  more  significant 
results  may  be  readily  obtained  in  the  future.   We  do  not  feel  that 
previous  applied  research  has  been  on  a  sound  theoretical  basis,  and 
we  hope  that  our  work  here  has  established  that  basis.   With  the 
development  of  a  theory  of  state  variable  inequality  constraints 
(SVIC's)  for  differential  games  we  feel  that  the  capabilities  for 
solving  applied  problems  would  be  appreciably  increased.   Therefore, 
we  give  this  task  top  priority. 

Based  on  our  past  research  experience  we  feel  that  there  is 
much  to  be  accomplished  in  the  future.   Specifically,  we  suggest  the 
following  as  future  research  tasks: 

(a)  Development  of  a  theory  of  state  variable  inequality  constraints 
for  Lanchester-type  differential  games. 

(b)  Development  of  further  results  for  deterministic  Lanchester 
one-versus-n-target-types  combat.   This  work  would  build  upon 
past  research  results  and  would  include 

(1)  development  of  more  complete  solutions  (for  cases  of  both 
constant  attrition-rate  coefficients  and  also  variable 
attrition-rate  coefficients) , 

(2)  incorporation  into  the  models  of  such  additional  factors 
as  replacements,  logistics,  and  supporting  weapons, 

(3)  further  consideration  of  "linear-law"  attrition  of 
target  types. 
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(c)  Study  of  the  optimal  division  of  forces  to  engage  target 
types.   A  slightly  different  attrition  model  than  in  the 
Isbell-Marlow  fire  distribution  problem  is  one  in  which  the 
friendly  forces  (assumed  homogeneous)  only  suffer  casualties 
when  they  engage  an  enemy  target  type.   (In  the  Isbell-Marlow 
problem  the  friendly  forces  sustain  casualties  from  an  enemy 
target  type  regardless  of  whether  or  not  they  engage  it.) 
This  model  is  especially  important,  since  it  can  be  used  to 
answer  several  questions  that  have  arisen  in  the  work  of  P. 
Chaiken. 

(d)  Development  of  simple  (probably  non-optimal)  fire  distribution 
policies  as  approximations  to  the  optimal  policy.   The  worth 
of  such  an  approximate  optimal  policy  would  be  evaluated  by 
comparing  the  payoff  associated  with  it  to  the  optimal  return. 
Besides  the  intrinsic  importance  of  this  aspect,  we  suggest 
this  task,  since  we  have  developed  complete  solutions  to 
several  deterministic  Lanchester  one-versus-two-target-type 
combat  problems. 

(e)  Examination  of  other  one-sided  allocation  problems  of  tactical 
interest  in  the  Lanchester  theory  of  combat  (for  example,  the 
optimal  distribution  of  supporting  fires  (such  as  Naval  fire 
support)  over  enemy  units  in  Lanchester  combat  with  friendly 
forces) .   Emphasis  would  be  on  solution  development  for  each 
of  various  modelling  alternatives  in  order  to  study  the 
dependence  of  optimal  tactics  on  model  form. 

(f)  Performance  of  computational  studies  on  the  optimal  control 
of  the  Lanchester  stochastic  process.   Using  the  finite 
difference  approximation  developed  in  Appendix  I,  we  would 
compare  optimal  strategies  for  deterministic  and  stochastic 
Lanchester  processes.   Furthermore,  we  would  develop  the 
fundamental  functional  equation  for  the  terminal  control 
battle  (in  Appendix  I  we  consider  the  prescribed  duration 
battle)  and  again  make  such  a  comparison.   Theoretical  inves- 
tigations should  concentrate  on  gaining  more  insight  into  the 
structure  of  the  optimal  target  selection  policy  when  the 
attrition  mechanism  is  stochastic. 

(g)  Further  study  of  the  optimal  control  of  deterministic  Lan- 
chester processes  when  there  is  a  bound  on  the  rate  of  change 
of  the  distribution  of  fire.   Results  of  our  initial  effort 
on  this  topic  are  contained  in  Appendix  J.   We  suggest  exten- 
sion of  these  preliminary  results. 
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(h)   Further  study  of  the  dependence  of  the  optimal  fire  distribution 
policy  on  the  nature  of  the  attrition  model.   We  have  given 
some  preliminary  results  in  Appendix  K  for  Helmbold's  general 
attrition  structure.   We  suggest  further  extension  of  this 
work  and  consideration  of  still  other  attrition  models  (for 
example,  that  of  Willard  [131]). 

(i)   Examination  of  optimal  tactics  for  other  objective  functions 

(i.e.  performance  (effectiveness)  criteria).   For  all  the  work 
contained  in  this  report  the  objective  function  has  been  a 
linear  function  of  the  number  of  survivors.   Other  objective 
functions  that  might  be  considered  are 

(1)  nonlinear  valuation  of  survivors  (such  as  quadratic  as 
motivated  by  Lanchester's  square  law), 

(2)  the  value  of  losses  rather  than  survivors. 

(j)   Consideration  of  some  simple  Lanchester-type  differential 
games.   All  previous  work  has  been  handicapped  by  the  lack 
of  an  adequate  theory  of  SVIC's  for  differential  games.   Such 
a  theory  can  be  used  to  reduce  the  volume  of  allocation 
strategy  combinations  that  must  be  considered.   Previous 
work  in  the  literature  for  which  the  non-negativity  restric- 
tions on  the  force  levels  has  not  received  adequate  treatment 
is  as  follows: 

(1)  the  war  of  attrition  and  attack  of  R.  Isaacs  [72], 

(2)  the  supporting  weapon  system  game  of  H.  K.  Weiss  [130], 

(3)  the  logistics  allocation  game  of  Moglewer  and  Payne  [95] , 

(4)  the  "two-on-two"  differential  game  studied  by  S.  Sternberg 
[111]- 

After  the  development  of  a  theory  of  SVIC's  for  Lanchester 
differential  games,  it  would  seem  appropriate  to  re-examine 
the  above  work.   Results  from  studying  these  problems  are 
useful  to  the  work  of  P.  Chaiken. 

(k)   Study  of  the  optimal  tactics  for  the  air  war  in  the  case  when 

the  air  war  is  evaluated  in  the  context  of  ground  war  objectives. 

Further  details  on  proposed  future  work  is  to  be  found  in  the 

discussion  sections  of  Appendices  E,  G,  I,  J,  and  K. 


68 


REFERENCES 


[1]   K.  Arrow  and  M.  Kurz,  "Optimal  Growth  with  Irreversible  Investment 
in  a  Ramsey  Model,"  Econometrica  38,  331-334  (1970). 

[2]   K.  Arrow  and  M.  Kurz,  Public  Investment,  The  Rate  of  Return,  and 
Optimal  Fiscal  Policy,  Johns  Hopkins  Press,  Baltimore  (1970). 

[3]   M.  Athans,  "The  Status  of  Optimal  Control  Theory  and  Applications 
for  Deterministic  Systems,"  IEEE  Trans,  on  Automatic  Control,  Vol. 
AC -11,  580-596  (1966)  . 

[4]   M.  Athans  and  P.  Falb ,  Optimal  Control,  McGraw-Hill,  New  York  (1966). 

[5]   R.  Bach,  L.  Dolansky  and  H.  Stubbs,  "Some  Recent  Contributions  to 
the  Lanchester  Theory  of  Combat,"  Opns .  Res.  10,  314-326  (1962). 

[6]   A.  Balakrishnan  and  L.  Neustadt  (Eds.),  Mathematical  Theory  of 
Control ,  Academic  Press,  New  York  (1967). 

[7]   C.  Barfoot,  "The  Lanchester  Attrition-Rate  Coefficient:   Some 

Comments  on  Seth  Bonder's  Paper  and  a  Suggested  Alternate  Method," 
Opns.  Res.  17,  888-894  (1969) . 

[8]   R.  Bellman,  Dynamic  Programming ,  Princeton  University  Press, 
Princeton  (1957) . 

[9]   L.  Berkovitz,  "On  Control  Problems  with  Bounded  State  Variables," 
J.  Math.  Anal.  Appl.  5,  488-498  (1962) . 

[10]   L.  Berkovitz,  "A  Variational  Approach  to  Differential  Games," 

pp.  127-174  in  Advances  in  Game  Theory,  M.  Dresher,  L.  Shapley, 
and  A.  Tucker  (Eds.),  Princeton  University  Press,  Princeton, 
N.  J.  (1964) . 

[11]   L.  Berkovitz,  "A  Differential  Game  with  No  Pure  Strategy  Solution," 
pp.  175-194  in  Advances  in  Game  Theory,  M.  Dresher,  L.  Shapley, 
and  A.  Tucker  (Eds.),  Princeton  University  Press,  Princeton, 
N.J.  (1964). 

[12]   L.  Berkovitz,  "Necessary  Conditions  for  Optimal  Strategies  in  a 

Class  of  Differential  Games  and  Control  Problems,"  SIAM  J.  Control  5, 
1-24  (1967). 

[13]   L.  Berkovitz,  "A  Survey  of  Differential  Games,"  pp.  342-372  in 

Mathematical  Theory  of  Control,  A.  Balakrishnan  and  L.  Neustadt  (Eds.), 
Academic  Press,  New  York  (1967). 


69 


[14]   L.  Berkovitz  and  M.  Dresher,  "A  Game-Theory  Analysis  of  Tactical 
Air  War,"  Opns.  Res.  7,  599-620  (1959). 

[15]   L.  Berkovitz  and  S.  Dreyfus,  "The  Equivalence  of  Some  Necessary 
Conditions  for  Optimal  Control  of  Problems  with  Bounded  State 
Variables,"  J.  Math.  Anal.  Appl.  10,  275-283  (1965). 

[16]   A.  Bishop  and  G.  Clark,  The  Tank  Weapon  System,  Report  No.  RF573AR69-2(U) , 
Systems  Research  Group,  The  Ohio  State  University  (1969). 

[17]   A.  Blaquiere,  F.  Gerard  and  G.  Leitman,  Quantitative  and  Qualitative 
Games ,  Academic  Press,  New  York  (1969). 

[18]   B.  Boltyanskii,  Mathematical  Methods  of  Optimal  Control,  Holt, 
Rinehart  and  Winston,  New  York  (1971). 

[19]   V.  Boltyanskii,  R.  Gamkrelidze  and  L.  Pontryagin,  "On  the  Theory  of 
Optimal  Processes,"  Reports  of  the  Academy  of  Sciences  of  the  USSR 
110,  7-10  (1956) (translation  reprinted  in  R.  Bellman  and  R.  Kalaba 
(Eds.),  Mathematical  Trends  in  Control  Theory,  Dover,  New  York  (1964)). 

[20]   S.  Bonder,  "Combat  Model,"  Chapter  2  in  The  Tank  Weapon  System, 

Report  No.  RF573AR64-1(U) ,  Systems  Research  Group,  The  Ohio  State 
University  (1964) . 

[21]   S.  Bonder,  "A  Theory  for  Weapon  System  Analysis,"  Proceedings 
U.S.  Army  Operations  Research  Symposium,  111-128  (1965) . 

[22]   S.  Bonder,  "The  Lanchester  Attrition-Rate  Coefficient,"  Opns.  Res.  15, 
221-232  (1967). 

[23]   S.  Bonder,  "The  Mean  Lanchester  Attrition  Rate,"  Opns.  Res.  18, 
179-181  (1970). 

[24]   S.  Bonder  and  R.  Farrell  (Eds.),  Development  of  Models  for  Defense 
Systems  Planning,  Report  No.  SRL2147TR70-2(U) ,  Systems  Research 
Laboratory,  The  University  of  Michigan  (1970) . 

[25]   M.  Boulton,  N.  Hopkins,  J.  Fain  and  W.  Fain,  "Comparing  Results  from 
a  War  Game  and  a  Computer  Simulation,"  pp.  739-755  in  Proceedings 
of  the  Fourth  International  Conference  on  Operational  Research, 
D.  Hertz  and  J.  Melese  (Eds.),  Wiley-Interscience,  New  York  (1966). 

[26]   H.  Brackney,  "The  Dynamics  of  Military  Combat,"  Opns.  Res.  7, 
30-44  (1959). 

[27]   R.  H.  Brown,  "Theory  of  Combat:   The  Probability  of  Winning," 
Opns.  Res.  11,  418-425  (1963). 


70 


[28]   A.  Bryson  and  Y.  C.  Ho,  Applied  Optimal  Control,  Blaisdell  Publishing 
Company,  Waltham,  Massachusetts  (1969). 

[29]   A.  Bryson,  W.  Denham  and  S.  Dreyfus,  "Optimal  Programming  Problems 
with  Inequality  Constraints  I:   Necessary  Conditions  for  Extremal 
Solutions,"  AIAA  Journal  1,  2544-2550  (1963). 

[30]   M.  Cannon,  C.  Cullum,  and  E.  Polak,  Theory  of  Optimal  Control  and 
Mathematical  Programming,  McGraw-Hill,  New  York  (1970). 

[31]   P.  Chaiken,  "Game-Theoretic  Analysis  of  Military  Operations — 

Description  of  the  Battlefield  Model,"  Final  Report,  Naval  Warfare 
Research  Center,  Stanford  Research  Institute  (1968). 

[32]   P.  Chaiken,  "An  Example  of  Force  Requirements  Calculations  Employing 
Game-Theoretic  Analysis  of  Military  Operations  (U),"  Final  Report, 
Naval  Warfare  Research  Center,  Stanford  Research  Institute  (1968) 
(SECRET) . 

[33]   P.  Chaiken,  "A  Force  Level  Requirements  Analysis  of  a  Two-Front  War 
(U),"  Research  Report,  Naval  Warfare  Research  Center,  Stanford 
Research  Institute  (1969) (SECRET) . 

[34]  P.  Chaiken,  "Operational  Application  of  Game  Theoretic  Models  (U)," 
Technical  Note,  NWRC  TN-28,  Naval  Warfare  Research  Center,  Stanford 
Research  Institute  (1969) (SECRET) . 

[35]   P.  Chaiken,  "A  Tactical  Analysis  of  the  Quick-Response  Collective 
Defense  Type  Contingency,"  unpublished  working  paper  dated  March 
1972. 

[36]   P.  Chaiken,  Personal  Communication,  October  1972. 

[37]   P.  Chaiken,  "General  Purpose  Forces  Trade-Off  Analyses  Involving  the 
Sea  War  Using  the  Balanced  Force  Requirement  Analysis  Model-BALFRAM," 
Final  Report,  Naval  Warfare  Research  Center,  Stanford  Research 
Institute,  to  be  published. 

[38]   R.  Chattopadhyay,  "On  Differential  Games,"  Int.  J.  Control  6, 
287-295  (1967). 

[39]   R.  Chattopadhyay,  "Differential  Game  Theoretic  Analysis  of  a  Problem 
of  Warfare,"  Naval  Res.  Log.  Quart.  16,  435-441  (1969). 

[40]  S.  Citron,  Elements  of  Optimal  Control,  Holt,  Rinehart  and  Winston, 
New  York  (1969) . 


71 


[41]   G.  Clark,  The  Combat  Analysis  Model,  Ph.D.  Dissertation,  The  Ohio 
State  University  (1968). 

[42]  M.  Conners  and  D.  Teichroew,  Optimal  Control  of  Dynamic  Operations 
Research  Models,  International  Textbook  Co.,  Scranton,  Pa.  (1967). 

[43]   F.  Dashiell  and  W.  Fain,  "Solution  of  the  Extended  Lanchester 
Equations  Used  in  a  Tactical  Warfare  Simulation  Programme," 
Canadian  Operational  Research  Society  Journal  4,  89-96  (1966) . 

[44]   B.  Davis  and  D.  J.  Elzinga,  "The  Solution  of  an  Optimal  Control 
Problem  in  Financial  Modeling,"  Opns .  Res.  19,  1419-1433  (1971). 

[45]   M.  Denn,  Optimization  by  Variational  Methods,  McGraw-Hill,  New  York 
(1969) . 

[46]   L.  Dolansky,  "Present  State  of  the  Lanchester  Theory  of  Combat," 
Opns.  Res.  12,  344-358  (1964). 

[47]   S.  Dreyfus,  "Variational  Problems  with  Inequality  Constraints," 
J.  Math.  Anal.  Appl.  4,  297-308  (1962) . 

[48]   J.  Engel  (Ed.),  Recent  Developments  in  Lanchester  Theory,  John 
Wiley,  New  York,  to  be  published. 

[49]   D.  Etter,  "Deterministic  Combat  Attrition  Models  for  Spatially 
Distributed  Forces,"  P.  577,  Institute  for  Defense  Analysis, 
Arlington,  Virginia  (1971). 

[50]   W.  Fain,  J.  Fain  and  H.  Karr,  "A  Tactical  Warfare  Simulation  Program,' 
Naval  Res.  Log.  Quart.  13,  413-436  (1966) . 

[51]   L.  Fan,  The  Continuous  Maximum  Principle,  John  Wiley  and  Sons,  Inc., 
New  York  (1966) . 

[52]   A.  Friedman,  Differential  Games,  Wiley-Interscience,  New  York  (1971). 

[53]  J.  Funk  and  E.  Gilbert,  "Some  Sufficient  Conditions  for  Optimality 
in  Control  Problems  with  State  Space  Constraints,"  SIAM  J.  Control 
8 ,  498-504  (1970) . 

[54]   B.  Goh,  "Necessary  Conditions  for  Singular  Extremals  Involving 
Multiple  Control  Variables,"  SIAM  J.  Control  4,  716-731  (1966). 

[55]   T.  Guinn,  "The  Problem  of  Bounded  Space  Coordinates  as  a  Problem  of 
Hestenes,"  SIAM  J.  Control  3,  181-190  (1965). 


72 


[56]   G.  Hall,  "Lanchester  Theory  of  Combat:   The  State  of  the  Art  in 

Mid-1970,"  M.S.  Thesis  in  Operations  Research,  Naval  Postgraduate 
School,  Monterey,  California  (1971) (available  through  DDC) . 

[57]   R.  Helmbold,  "Lanchester  Parameters  for  Some  Battles  of  the  Last 
Two  Hundred  Years,"  Combat  Operations  Research  Group,  Technical 
Operations,  Inc.,  CORG-SP-122,  February  1961. 

[58]   R.  Helmbold,  "Some  Observations  on  the  Use  of  Lanchester' s  Theory 
for  Prediction,"  Opns .  Res.  12,  778-781  (1964). 

[59]   R.  Helmbold,  "A  Modification  of  Lanchester's  Equations,"  Opns.  Res. 
13,  857-859  (1965). 

[60]   0.  Helmer,  "Combat  Between  Heterogeneous  Forces,"  RM-6 ,  The  RAND 
Corporation  (1947) . 

[61]  M.  Hestenes,  "A  General  Problem  in  the  Calculus  of  Variations  with 
Applications  to  Paths  of  Least  Time,"  RM  100,  The  RAND  Corporation 
(1949). 

[62]   M.  Hestenes,  Calculus  of  Variations  and  Optimal  Control  Theory, 
John  Wiley  and  Sons,  Inc.,  New  York  (1966). 

[63]   Y.  C.  Ho,  "Review  of  the  Book  Differential  Games  by  R.  Isaacs," 
IEEE  Trans,  on  Automatic  Control,  Vol.  AC-10,  501-503  (1965) . 

[64]   Y.  C.  Ho,  "Toward  Generalized  Control  Theory,"  IEEE  Trans,  on 
Automatic  Control,  Vol.  AC-14 ,  753-754  (1969). 

[65]   Y.  C.  Ho,  A.  Bryson  and  S.  Baron,  "Differential  Games  and  Optimal 
Pursuit-Evasion  Strategies,"  IEEE  Trans,  on  Automatic  Control, 
Vol.  AC-10,  385-389  (1965). 

[66]  M.  Intriligator ,  "Strategy  in  a  Missile  War:  Targets  and  Rates  of 
Fire,"  Security  Studies  Paper  Number  10,  University  of  California, 
Los  Angeles  (1967) . 

[67]   M.  Intriligator,  Mathematical  Optimization  and  Economic  Theory, 
Prentice-Hall,  Englewood  Cliffs,  N.  J.  (1971). 

[68]   R.  Isaacs,  "Differential  Games  I:   Introduction,"  RM-1391,  The  RAND 
Corporation  (1954) . 

[69]   R.  Isaacs,  "Differential  Games  II:   The  Definition  and  Formulation," 
RM-1399,  The  RAND  Corporation  (1954). 


73 


[70]   R.  Isaacs,  "Differential  Games  III:   The  Basic  Principles  of  the 
Solution  Process,"  RM-1411,  The  RAND  Corporation  (1954). 

[71]   R.  Isaacs,  "Differential  Games  IV:   Mainly  Examples,"  RM-1486,  The 
RAND  Corporation  (1955) . 

[72]   R.  Isaacs,  Differential  Games,  John  Wiley  and  Sons,  Inc.,  New  York 
(1965) . 

[73]   J.  Isbell  and  W.  Marlow,  "Attrition  Games,"  Naval  Res.  Log.  Quart.  3, 
71-94  (1956). 

[74]   J.  Isbell  and  W.  Marlow,  "Methods  of  Mathematical  Tactics," 

Logistics  Papers,  No.  14,  The  George  Washington  University  Logistics 
Research  Project,  September  1956. 

[75]   D.  Jacobson,  "A  New  Necessary  Condition  of  Optimality  for  Singular 
Control  Problems,"  SIAM  J.  Control  7,  578-595  (1969). 

[76]   D.  Jacobson,  "A  General  Sufficiency  Theorem  for  the  Second  Variation," 
J.  Math.  Anal.  Appl.  34,  578-589  (1971). 

[77]   D.  Jacobson  and  M.  Lele,  "A  Transformation  Technique  for  Optimal 
Control  Problems  with  a  State  Variable  Inequality  Constraint," 
IEEE  Trans.  Automatic  Control,  Vol.  AC-14,  457-464  (1969) . 

[78]   D.  Jacobson  and  J.  Speyer,  "Necessary  and  Sufficient  Conditions  for 
Optimality  for  Singular  Control  Problems:   A  Limit  Approach," 
J.  Math.  Anal.  Appl.  34,  239-266  (1971). 

[79]   D.  Jacobson,  M.  Lele  and  J.  Speyer,  "New  Necessary  Conditions  of 
Optimality  for  Control  Problems  with  State-Variable  Inequality 
Constraints,"  J.  Math.  Anal.  Appl.  35,  255-284  (1971). 

[80]   C.  Johnson,  "Singular  Solutions  in  Problems  of  Optimal  Control," 

pp.  209-267  in  Advances  in  Control  Systems,  Vol.  2,  C.  Leondes  (Ed.), 
Academic  Press,  New  York  (1965). 

[81]   C.  Johnson  and  J.  Gibson,  "Singular  Solutions  in  Problems  of 

Cptimal  Control,"  IEEE  Trans,  on  Automatic  Control,  Vol.  AC-8, 
4-15  (1963) . 

[82]   H.  Kelley,  "A  Second  Variational  Test  for  Singular  Extremals," 
AIAA  Journal  2,  1380-1382  (1964) . 

[83]   H.  Kelley,  R.  Kopp  and  H.  Moyer ,  "Singular  Extremals,"  pp.  63-101 
in  Topics  in  Optimization,  G.  Leitman  (Ed.),  Academic  Press,  New 
York  (1967) . 


74 


[84]   S.  Kimbleton,  "Attrition  Rates  for  Weapons  with  Markov-Dependent 
Fire,"  Opns.  Res.  19,  698-706  (1971). 

[85]   T.  Kisi  and  T.  Hirose,  "Winning  Probability  in  an  Ambush  Engagement," 
Opns.  Res.  14,  1137-1138  (1966)  . 

[86]   B.  Koopman,  "A  Study  of,  the  Logical  Basis  of  Combat  Simulation," 
Opns.  Res.  18,  855-882  (1970)  . 

[87]   F.  W.  Lanchester,  Aircraft  in  Warfare;  The  Dawn  of  the  Fourth  Arm, 
Constable,  London  (1916)  . 

[88]   F.  Lanchester,  "Mathematics  in  Warfare,"  pp.  2138-2157  in  The  World 
of  Mathematics,  Vol.  IV,  J.  Newman  (Ed.),  Simon  and  Schuster,  New 
York  (1956) . 

[89]   E.  Lee  and  L.  Markus ,  Foundations  of  Optimal  Control  Theory,  John 
Wiley  and  Sons,  Inc.,  New  York  (1967). 

[90]   G.  Leitmann  (Ed.),  Optimization  Techniques,  Academic  Press,  New 
Press,  New  York  (1962) . 

[91]   G.  Leitmann  (Ed.),  Topics  in  Optimization,  Academic  Press,  New  York 
(1967). 

[92]   0.  Mangasarian,  "Sufficient  Conditions  for  the  Optimal  Control  of 
Nonlinear  Systems,"  SIAM  J.  Control  4,  139-152  (1966). 

[93]   0.  Mangasarian  and  S.  Fromovitz,  "A  Maximum  Principle  in  Mathematical 
Programming,"  pp.  85-95  in  Mathematical  Theory  of  Control,  A.  Bala- 
krishnan  and  L.  Neustadt  (Eds.),  Academic  Press,  New  York  (1967). 

[94]   J.  Mclntyre  and  B.  Paiewonsky,  "On  Optimal  Control  with  Bounded 
State  Variables,"  pp.  389-419  in  Advances  in  Control  Systems, 
Vol.  5,  C.  Leondes  (Ed.),  Academic  Press,  New  York  (1967). 

[95]   S.  Moglewer  and  C.  Payne,  "A  Game  Theory  Approach  to  Logistics 
Allocation,"  Naval  Res.  Log.  Quart.  17,  87-97  (1970). 

[96]   P.  Morse  and  G.  Kimball,  Methods  of  Operations  Research,  M.I.T. 
Press,  Cambridge,  Massachusetts  (1951). 

[97]   T.  Oberbeck,  "A  Note  on  the  Lanchester  Equations,"  RM-12,  The  RAND 
Corporation  (1947) . 

[98]   T.  Oberbeck,  "A  Second  Note  on  the  Lanchester  Equations,"  RM-13, 
The  RAND  Corporation  (1947) . 


75 


[99]   B.  Paiewonsky,  "Optimal  Control:   A  Review  of  Theory  and  Practice," 
AIAA  Journal  3,  1985-2006  (1965) . 

[100]   E.  Paxson,  "War  Gaming,"  RM-3489-PR,  The  RAND  Corporation  (1963). 

[101]   L.  Pontryagin,  V.  Boltyanskii,  R.  Gamkrelidze  and  E.  Mishchenko , 
The  Mathematical  Theory  of  Optimal  Processes,  Interscience,  New 
York  (1962). 

[102]   I.  B.  Russak,  "On  Problems  with  Bounded  State  Variables," 
J.  Opt.  Th.  App.  5,  114-157  (1970) . 

[103]  J.  Rustagi  and  R.  Laitinen,  "Moment  Estimation  in  a  Markov-Dependent 
Firing  Distribution,"  Opns .  Res.  18,  918-923  (1970). 

[104]   J.  Rustagi  and  R.  Srivastava,  "Parameter  Estimation  in  a  Markov- 
Dependent  Firing  Distribution,"  Opns.  Res.  16,  1222-1227  (1968). 

[105]   T.  Saaty,  Mathematical  Models  of  Arms  Control  and  Disarmament, 
John  Wiley  &  Sons,  Inc.,  New  York  (1968). 

[106]   K.  Shell  (Ed.),  Essays  on  the  Theory  of  Optimal  Economic  Growth, 
M.I.T.  Press,  Cambridge,  Massachusetts  (1967). 

[107]  E.  Simakova,  "Differential  Games,"  Automation  and  Remote  Control  27, 
1980-1998  (1967) (English  translation  from  Avtomatika  i  Telemekhanika 
_2_7_,  161-178  (1966)  . 

[108]   D.  Smith,  "The  Probability  Distribution  of  the  Number  of  Survivors 
in  a  Two-Sided  Combat  Situation,"  Operational  Res.  Quarterly  16, 
429-437  (1965). 

[109]   R.  Snow,  "Contributions  to  Lanchester  Attrition  Theory,"  The  RAND 
Corporation,  Report  RA-15078  (1948). 

[110]   J.  Speyer  and  D.  Jacobson,  "Necessary  and  Sufficient  Conditions  for 

Optimality  for  Singular  Control  Problems;  A  Transformation  Approach," 
J.  Math.  Anal.  Appl.  33,  163-187  (1971). 

[Ill]   S.  Sternberg,  "Development  of  Optimal  Allocation  Strategies  in 

Heterogeneous  Lanchester-Type  Processes,"  Report  No.  SRL  2147  TR  71-1 
(U) ,  Systems  Research  Laboratory,  The  University  of  Michigan  (1971) . 

[112]   J.  Taylor,  "Application  of  Differential  Games  to  Problems  of  Naval 
Warfare:   Surveillance  -  Evasion  -  Part  I,"  Naval  Postgraduate 
School  Technical  Report  No.  NPS55TW0061A,  19  June  1970. 


76 


[113]   J.  Taylor,  "Application  of  Differential  Games  to  Problems  of 

Military  Conflict:   Tactical  Allocation  Problems  -  Part  1,"  Naval 
Postgraduate  School  Technical  Report  No.  NPS55TW0062A,  19  June  1970. 

[114]   J.  Taylor,  "A  Note  on  the  Solution  to  Lanchester-Type  Equations 
with  Variable  Coefficients,"  Opns .  Res.  19,  709-712  (1971). 

[115]  J.  Taylor,  "Comments  on  a  Multiplier  Condition  for  Problems  with 
State  Variable  Inequality  Constraints,"  IEEE  Trans,  on  Automatic 
Control,  Vol.  AC-17,  743-744  (1972)  . 

[116]   J.  Taylor,  "Comments  on  'A  Note  on  the  Solution  to  Lanchester-Type 
Equations  with  Variable  Coefficients,1"  Opns.  Res.  20,  to  appear. 

[117]   J.  Taylor,  "On  the  Isbell  and  Marlow  Fire  Programming  Problem," 
Naval  Res.  Log.  Quart.,  to  appear. 

[118]   J.  Taylor,  "A  Squared-Variable  Transformation  Approach  to  Nonlinear 

Programming  Optimality  Conditions,"  Naval  Res.  Log.  Quart.,  to  appear. 

[119]   J.  Taylor,  "Lanchester-Type  Models  of  Warfare  and  Optimal  Control," 
(tentatively)  accepted  for  publication  in  Naval  Res.  Log.  Quart. 

[120]   J.  Taylor,  "Target  Selection  in  Lanchester  Combat:   Heterogeneous 
Forces  and  Time  Dependent  Attrition  Rates,"  (tentatively)  accepted 
for  publication  in  Naval  Res.  Log.  Quart. 

[121]   J.  Taylor,  "Target  Selection  in  Lanchester  Combat:   Linear-Law 

Attrition  Process,"  (tentatively)  accepted  for  publication  in  Naval 
Res.  Log.  Quart. 

[122]   J.  Taylor,  "Comments  on  Some  Differential  Games  of  Tactical  Interest," 
submitted  for  publication  to  Opns.  Res. 

[123]   J.  Taylor,  "On  the  Solution  to  Lanchester-Type  Equations  of  Modern 
Warfare  with  Variable  Coefficients,"  submitted  for  publication  to 
Opns.  Res,  (also  presented  at  39th  Opns.  Res.  Soc.  of  Am.  Meeting, 
5-7  May  1971)  . 

[124]   C.  Thomas  and  W.  Deemer,  "The  Role  of  Operational  Gaming  in  Operations 
Research,"  Opns.  Res.  5,  1-27  (1957). 

[125]   F.  Valentine,  "The  Problem  of  Lagrange  with  Differential  Inequalities 
as  Add  Side  Conditions,"  pp.  403-447  in  Contributions  to  the  Calculus 
of  Variations,  1933-1937,  The  University  of  Chicago  Press,  Chicago, 
Illinois  (1937) . 


77 


[126]   P.  Wallis,  "Recent  Developments  in  Lanchester  Theory,"  Operational 
Res.  Quart.  19,  191-195  (1968). 

[127]   G.  Weiss,  "Comparison  of  a  Deterministic  and  a  Stochastic  Model 
for  Interaction  Between  Antagonistic  Species,"  Biometrics  19, 
595-602  (1963)  . 

[128]   H.  K.  Weiss,  "Requirements  for  a  Theory  of  Combat;  Lanchester  Models," 
BRL  Report  No.  667  (1953). 

[129]   H.  K.  Weiss,  "Lanchester-Type  Models  of  Warfare,"  Proc.  First 
International  Cont .  Operational  Res. ,  Oxford  (1957) . 

[130]   H.  K.  Weiss,  "Some  Differential  Games  of  Tactical  Interest  and  the 
Value  of  a  Supporting  Weapon  System,"  Opns .  Res.  7,  180-196  (1959). 

[131]   D.  Willard,  "Lanchester  as  Force  in  History:   An  Analysis  of  Land 
Battles  of  the  Years  1618-1905,"  Research  Analysis  Corporation 
RAC-TP-74,  November  1962. 


Appendix  A.   On  the  Isbell  and  Marlow  Fire  Programming  Problem. 

1.   Introduction. 

In  this  appendix  a  complete  solution  is  derived  to  the  Isbell 
and  Marlow  fire  programming  problem.   The  original  work  of  Isbell 
and  Marlow  has  been  extended  by  determining  the  regions  of  the 
initial  state  space  from  which  optimal  paths  lead  to  each  of  the 
terminal  states  of  combat.   The  solution  process  has  involved 
determining  the  domain  of  controllability  for  each  of  the  terminal 
states  of  combat  and  the  determination  of  dispersal  surfaces.   This 
solution  process  suggests  a  solution  procedure  applicable  to  a  wider 
class  of  tactical  allocation  problems,  terminal  control  attrition 
differential  games.   The  structure  of  optimal  target  engagement 
policies  in  "fights  to  the  finish"  is  discussed. 

An  underdeveloped  area  [6]  of  the  Lanchester  theory  of  combat 
is  target  selection  for  combat  among  heterogeneous  forces.   This  type 
of  problem  has  been  studied  by  Isbell  and  Marlow,  who  considered  both 
a  truncated  stochastic  (Lanchester)  process  by  game  theoretic  means 
[12]  and  a  terminal  control  (one-sided)  differential  game  [11].   An 
attrition  differential  game  is  an  idealized  combat  situation  described 
by  Lanchester-type  equations  [5],  [18]  over  a  period  of  time  with 
choices  of  tactics  available  to  both  sides  and  subject  to  change  with 
time.   Terminal  control  attrition  games  only  end  when  the  course  of 
combat  has  been  steered  to  a  prescribed  state. 
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In  developing  a  theory  of  target  selection  it  is  important 
to  understand  the  dependence  of  allocation  rules  on  the  type  of  model 
chosen.   Tactical  allocation  problems  may  be  studied  in  two  types  of 
scenarios:   (1)  the  prescribed  duration  battle  and  (2)  the  terminal 
control  battle  (a  particular  case  of  which  is  the  "fight  to  the 
finish").   All  the  attrition  examples  in  Isaacs'  book  [14]  are  of  the 
first  type  (his  "War  of  Attrition  and  Attack"  is  the  continuous 
version  of  the  tactical  air  war  game  [3],  [4],  [9]  studied  at  RAND). 
Only  Isbell  and  Marlow  [11]  and  Weiss  [19]  have  studied  the  terminal 
control  problem.   Unfortunately,  Isbell  and  Marlow  did  not  obtain 
a  complete  solution  to  their  problem.   They  could  not  dete -nine  when 
certain  terminal  states  of  combat  were  reached.   Weiss  studied  a  problem 
which  may  be  considered  to  be  a  generalization  (two-sided  version)  of 
their  problem.   Writing  many  years  before  results  were  known  beyond 
a  small  number  of  researchers,  he  did  not  use  the  subsequently  well- 
known  necessary  conditions  [1].   That  his  solution  procedure  [19]  was 
heuristic  is  not  surprising,  since  the  simpler  problem*  [11]  which 
he  referenced  in  his  paper  had  not  been  completely  solved. 


Since  the  original  version  of  this  paper,  this  problem  has  also  been 
recently  studied  by  S.  Sternberg  [17].   He  employed  a  more  geometric 
approach  closely  following  that  of  R.  Isaacs  [14]  and  originally 
employed  by  Isbell  and  Marlow  [11].  Sternberg  obtained  different 
results  than  those  presented  here. 
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Thus  the  obtaining  of  a  complete  solution  to  the  Isbell  and 
Marlow  problem  is  viewed  as  a  first  step  leading  to  the  solution  of 
more  general  tactical  allocation  problems.   In  this  present  appendix,  we 
shall  derive  its  solution  by  the  Pontryagin  maximum  principle  [16]. 
Generalizations  of  this  procedure  to  differential  games  are  indicated, 
and  solution  properties  are  discussed.   In  view  of  the  close  connec- 
tion [1],  [10]  between  optimal  control  and  differential  games  (Isaacs), 
the  terminology  of  these  two  fields  is  used  somewhat  interchangeably. 

2.   The  Fire  Distribution  Problem. 

The  situation  considered  by  Isbell  and  Marlow  [11]  is  the 
simplest  problem  of  fire  distribution:   combat  betwee   an  X-force  at 
two  force  types  (for  example,  riflemen  and  grenadiers)  and  a  homo- 
geneous Y-force  (for  example,  riflemen  only) .   This  situation  is 
shown  diagrammatically  below. 
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It  is  the  objective  of  the  Y-force  commander  to  maximize  his  survi- 
vors at  the  end  of  battle  and  minimize  those  of  his  opponent  (consid- 
ering the  utilities  assigned  survivors) .   This  is  accomplished  through 
his  choice  of  the  fraction  for  fire,  <P,      directed  at  X  .   The  battle 
terminates  when  one  side  or  the  other  has  been  annihilated. 
Mathematically  the  problem  may  be  stated  as 


maximize  ry(T)  -  px..  (T)  -  qx  (T)   with  T   unspecified 

dxl 

subject  to:  - —  =  -0a..  y 
at       I 

dx 

sf-  -  -<i-#>v 

ft      =  "Vl  "  b2X2  (1) 


x  ,x  ,y  ^  0   and   0  £  <p   £  1 


where  all  symbols  are  defined  in  the  next  section  and  with  terminal 
states  defined  by   (1)   x  (T)  =  x2(T)  -  0  and   (2)   y(T)  =  0. 

The  terminal  surface  of  the  "realistic"  (one-sided)  game  is 
seen  to  consist  of  five  parts: 


A- 5 


Cx:   Xl(T)  =  0,   x2(T)  >  0,   y(T)  =  0 


C2:   x1(T)  =  0   before   x^T)  =  0,   y(T)  >  0, 


C3:   x1(T)  =  0   after   x2(T)  =  0,   y(T)  >  0, 


C4:  x±(T)    >    0,   x2(T)  =  0,   y(T)  =  0, 


C5:   X;L(T)  >  0,   x2(T)  >  0,   y(T)  =  0. 


3.   Notation. 

The  symbols  which  are  used  in  this  paper  are  defined  as 
follows : 

A  =  A(R,z)  =  [z2(R-l)  -  R]/(z-l)2, 

B  =  B(R,z)  =  A(z-l)2/z2  =  [z2(R-l)  -  R]/z2, 

a  ,a  ,b  ,b   =  constant  attrition  rates, 

C.   for   i  =  1,2,3,4,5  =  the  i —  part  of  the  terminal  surface 

as  defined  in  section  2., 

D(C.)  =  domain  of  controllability  for   C, 

g(P  ,R,z)  =  term  in  equation  (57)  of  the  locus  of  points  for 
which   P   =  P, , 

h(P  ,R,z)  =  term  in  equation  (58)  for  boundary  surface  between 

the  regions  from  which  optimal  paths  lead  to   C    and   C  , 

p,q,r  =  utilities  assigned  to  surviving  X  ,   X   and  Y 
forces  respectively, 

p.(t)   for   i  =  1,2,3  =  dual  variable  corresponding  to  x.(t) 
1  (x3(t)  =  y(t)),  X 

p.   for   i  =  1,2,3  =  boundary  condition  for  dual  variable  at 
1  t=0, 
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P.   for   i  =  1,2,3,4,5  =  payoff  associated  with  an  extremal 

leading  to  C . , 

P  =  (x  ,x_,y_)  =  point  in  the  initial  state  space, 

R=alV(a2b2>> 

.00.  o  o 

s   =   s(x1,x2)    =  b   x1  +  t>2x2, 

t  =  time  at  which  X   is  annihilated,   i.e.   x  (t  )  =  0, 

t„  =  first  time  at  which   2b  x  (t Jx  +  b_(x  )2=  a  y2(t?)   for 
an  extremal  leading  to  C,  , 

T  =  total  time  for  the  battle, 

v  =  v(t)    =  a2P2(x)    -  a1p1(t), 

a,       (b.p.  +  b_p) 


=   cosh    /a  b      t-  (C.)    =  —er     7 — r 7— , 

2   2      14  p2      (a1b1   -  a2b2) 


•  -1         -1      00 
x  ,x  ,y  =  average  force  strengths;  with  initial  values   x  ,  x  , 

a,   (biq-b2p)     R_6 


z  =  cosh  /a_b   x  (C_)  =  —  - — 
2  2   15     q   (a. 


lbl  "  a2b2>    R  -  1 


6  =  a1p/(a2q) , 


0   =  fraction  of  Y-f ire  directed  at  X..  , 

t  =  "backwards  time"  from  the  end  of  the  reduced  game,  i.e. 
the  time  remaining  before  the  end  of  the  reduced  game, 

x  (C . )  =  "backwards  time"  of  the  first  switch  in  tactics  for 
extremals  leading  to   C., 

t1  =  T  -  t  =  length  of  time  that  Y   fires  at  X   after  X 
has  been  annihilated  for  C.   extremals. 


4 .   Solution  Procedure  and  Extensions. 

Extremal  paths  (a  path  on  which  the  necessary  conditions  for 
optimality  are  almost  everywhere  satisfied)  may  be  obtained  by  routine 
application  of  Pontryagin's  maximum  principle  [16]  (the  original 
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authors  used  equivalent  conditions  independently  developed  by  Isaacs 
[13]).   However,  in  a  terminal  control  problem  we  would  like  to  know 
the  domain  of  controllability  [7]  for  each  terminal  state  so  that 
tactics  are  determined  in  terms  of  the  initial  conditions  of  combat 
(and  also  possibly  time) .   We  define  the  domain  of  controllability 
for  a  given  terminal  state  to  be  that  subset  of  the  initial  state 
space  from  which  extremals  lead  to  the  terminal  state.   Furthermore, 
additional  considerations  (as  outlined  below)  are  required  when  the 
domains  of  controllability  corresponding  to  different  terminal  states 
overlap . 

The  following  procedure  has  been  used  to  solve  the  above 
problem: 

(a)  extremal  control  is  determined  by  maximizing  the  Hamiltonian; 
since  the  state  variables  (force  strengths)  are  non-negative, 

the  control  depends,  in  many  cases,  only  on  relationships  between 
the  dual  variables  (marginal  return  from  destroying  target) , 

(b)  from  each  separate  terminal  state,  the  time  history  of  the  dual 
variables  is  obtained  by  a  backward  integration  of  the  adjoint 
system  of  differential  equations;  for  a  square  law  attrition 
process,  the  adjoint  equations  are  independent  of  the  state 
variables , 

(c)  for  each  terminal  state  the  domain  of  controllability  is  deter- 
mined by  forward  integration  of  the  state  equations  using  the 
time  history  of  extremal  control  developed  in  (b) ;  changes  in 
control  with  time  (existence  of  transition  surface)  may  have  to 
be  considered  in  this  step. 
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(d)  the  solution  is  determined  at  this  point  for  regions  of  the 
initial  state  space  which  are  covered  by  part  of  the  domain  of 
controllability  for  only  one  terminal  state;  one  must  also 
verify  that  the  entire  initial  state  space  has  been  accounted 
for,  since  otherwise  one  may  have  overlooked  some  type  of 
"singular"  surface  or  even  a  terminal  state, 

(e)  if  domains  of  controllability  overlap  so  that  for  a  point  of 
the  initial  state  space  contained  in  their  intersection  there  is 
more  than  one  extremal  leading  to  the  terminal  surface,  then  one 
computes  the  payoff  associated  with  each  extremal;  the  optimal 
trajectory  is  selected  from  the  extremals  by  comparing  these 
values . 

It  is  noted  that  Isbell  and  Marlow  [11]  stopped  at  step  (b)  above. 

It  seems  appropriate  to  further  discuss  the  situation  when, 
for  example,  the  domains  of  controllability  corresponding  to  two 
different  terminal  states  overlap.   For  points  of  the  initial  state 
space  contained  in  the  intersection  of  the  two  domains,  there  are  two 
mutually  exclusive  possibilities:   (1)  for  every  point  the  payoff 
associated  with  the  extremal  leading  to  one  terminal  state  always 
dominates  that  corresponding  to  the  other,  or  (2)  the  payoff  corre- 
sponding to  extremals  of  one  family  does  not  always  dominate  that  of 
the  other.   In  the  first  case,  the  solution  is  simply  determined: 
the  optimal  trajectories  are  the  extremals  corresponding  to  the  more 
favorable  payoff.   In  the  second  case,  one  must  determine  for  what 
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parts  of  the  region  of  intersection  each  alternative  is  better.   For 
the  problem  at  hand,  it  will  be  seen  that  such  a  region  of  intersec- 
tion may  be  divided  into  two  subregions  separated  by  a  boundary  surface. 
In  each  subregion  the  payoff  corresponding  to  one  family  of  extremals 
dominates  that  of  the  other.   R.  Isaacs  (see  pp.  132-141  of  [14])  has 
referred  to  such  a  singular  surface  as  a  dispersal  surface,  since  when 
the  geometry  of  the  optimal  trajectories  is  considered,  optimal  paths 
only  lead  away  from  a  dispersal  surface.   For  the  problem  at  hand,  it 
will  be  seen  that  for  a  certain  range  of  model  parameters,  two  dis- 
persal surfaces  are  present  in  its  solution. 

The  complete  solution  to  this  problem  is  shown  in  Tables  I 
and  II  with  supporting  details  presented  in  the  next  section.   When 
a  p  ^  a  q,   the  domains  of  controllability  do  not  overlap  so  that  the 
extremals  are  unique,  and  the  extremal  control  turns  out  to  be  the 
optimal  control.   Hence,  the  solution  procedure  terminates  at  step  (d) 
above.   When  a  q  >  a  p,   the  domains  of  controllability  are  shown 


in  Table  I  as  Case  (2).   For  R  -  /R(R-l)  s:  6  <  1  where   6  =  (a  p)/(a  q). 
the  domains  of  controllability  don't  overlap  except  for  that  corresponding 
to  part   C,   of  the  terminal  surface,  which  overlaps  several  others. 
In  section  5.2  below  it  is  shown  how  optimal  trajectories  are  deter- 

ined  in  the  region  of  overlap  and  that  extremals  leading  to  C,   can 
only  be  optimal  for  a  by  2  *  s2  +  A(b  x  )2.   The  resulting  solution 
is  shown  in  Table  II  as  Case  (a).   It  should  be  noted  that  non-binding 
constraints,  which  define  the  regions  in  the  initial  state  space  for 


m 
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TABLE  I.   Solution  to  Target  Selection  Problem  -  Fight  to  Finish 

Nonrestrictive  assumption:  R  >  1,   i.e.   aibi  >   a?bo 

Terminal  State  Optimal  Control  Domain  of  Controllability 

Case  (1) :   a  q  £  a  p 

Xl(tl}    =   °  f\      for     0  £   t  £   t,  alblyo2   <    s2  +    (R-D(b2x2)2 


Cl  \    X2(T)    "    °  <P*(C) 


C2 


Case    (2) :      a   q    >    a   p 


/l      for      0  £   t  £   t- 
^0     for      t,   <  t  <  T 


v i aiVo2  * s2  -  (b2x°2)2 


S(tl>    =  °  /l     forOititj 

x2(T)    =0  «>*(t)    =   I                                                               albiyo2    >   s2  +    (R-l)(b2x2); 

y(T)    >   0  1°      f°r      Cl  *   C  *  T 

VT)    "   °  ahv   2    <    S2+    (R.1)(b      o); 


C5\    X2(T)    >   °  0*(t)    =   1      f°r      °  *   t  *  T 

y(T)  =  o 


Extremal  Control 


C  Same   as   Case    (1) 


1   l/o  "  v      ~'*"2 

aibiyo2  <  s2  -  (b2x2): 


albiyo2   <   s"  +    (R-D(b2X2)2 
a^y/   S  s2   +  B(b2x°)2 


C  Same  as  Case  (1)  Same  as  Case  (1) 

-l<t2)    >0  /i,for     PM*t2  alblyo2   <   s2  +   (R-D(b2x2)2 

x2(T)    =0  0*(t)    =<  albiyo2   *   R{s2   "    (b!XJ)2} 

y(T)=0  1°     f°r     t2*'t'*1  a1b1yo2*s2  +  R(R-l)(b2x;)' 

albiyo2    <    s2  +    (R"1)(b2X2)2 
C  0>*(t)    =0      for      0  ^   t  £  T  alblyo2    <    Rts2    ~    (bix°)2} 


1      for      0  £   t  £  T   -   t 


alVo2  *  Rs2{1   ~  I2"1 


alblyo2    <    s2  +    (R"1)(b2x2)2 
1  a1b1yQ2   <    s2  +  B(b2x°)2 


CS  *    (t)       A  ,  o,2 

'  m  a.b.y  2   >   s2  +  A(b.x  )^ 

0      for     T   -    t     £   t  £  T  iro 


Definition  of  Times 


(a)   t.   is  first   t   such  that  x  (t^)  =  0. 


aiVo2  *  Rs2{1  "  I7* 


(b)   t2   is  first   t   such  that   2b1x1(t2)x°  +  b2(x°)2  =  a2y2(t2). 


ax    (b1q-b2p) 


(c)      t1      is   determined  by      cosh    /afo   ^  =  —   <a  b^a^) 


TABLE  II.   Target  Selection  Problem  -  Fight  to  Finish 

Solution  for  a.q  >  a  p  and  Various  Values  of  Parameters 
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Nonrestrictive  assumption:   R  >  1,   i.e.   a..b   >  a  b 


Terminal  State 


Optimal  Control 


Case  (a)  :   A  £  0,   i.e.   R  -  /r(r-1)  £  6  <  1 


xn(T)  >  0 


y(T)  =  0 
rx1(t1)  =  0 

x2(T)  =  0 
^y(T)  >  0 
''x1(t2)  >  0 

x2(T)  =  0 

y(T)  =  o 


0*(t)  =• 


0*(t)  = 


0*(t)  = 


1  for  0  £  t  £  t, 


0  for  t.  <  t  s  T 


1  for  0  £  t  £  t, 


0   for   tj  i  t  «  T 


1  for  Octet. 


0  for   t2  ^  t  £  T 


Region  of  Initial  Force  Levels 


alVo2   <   s2  +    (R"1)(b2X2)2 
alVo2   *  s2  +  B^b2x2)2 


a.b.y   *   >   s2  +    (R-l)(b,x°)2 


*1    I/O 


22' 


alblyo2  *  R{s2  ~  (bixi)2} 
a.b.y  2  *  s2  +  A(b„x°)2 


'ri'o 


22' 


rx  (T)  >  0 
x2(T)  >  0 
y(T)    =   0 


0*(t) 


1      for      0  £   t  £  T 


0      for     T   -   t.    £  t  £  T 


a.b  y  2   >   s2  +  A(b,x°)2 
1    1   o  2   2 

alVo2   <   s2  +  B(b2x2)2 

anbny  2   >   Rs2{l  -  1/z2} 
1  1  o 


0*(t)  -  0   for  0  £  t  £  T 


alblyl2  *  Rs2{1  "  1/z2} 


Terminal  State    Optimal  Control 
;ase  (b) :  A  >  0,   i.e.   0  £  6  <  R  -  /R(R-l) 


Same  as  Case  (a) 


alblyo2  <  R{s2  "  (bixi)2} 
Region  of  Initial  Force  Levels 

alVo2  <  s2  +  (R"1)(b2x2)2 

alblyo2£  Rs2  "  R^b1x°[z2(R-l)+R]/(2R)  +  b2x2)2/z2 

for  0  £  x°  <  (b2x°)/(kb1) 
alblyo2  *  h(p°»R»z)   for  0  £  x°  £  kb1x°/b2 


Same  as  Case  (a) 


Same  as  Case  (a) 


Same  as  Case  (a) 


0*(t)  =  0   for  0  £  t  £  T 


alblyo2  <  s2  +  (R-1)(b2X2)2 

alblyo2  *  Rls2  "  (bixi)2} 

a  b  yQ2  £  h(P°,R,z)   for  0  £  x°  £  kb^x"^ 

a.b.y  2  <  s2  +  (R-l)(b_x°)2 
1  1  o  2  I 

a.b.y  2  <  R{s2  -  (b.x°)2} 
1  1  o  11 

a.b.y  2  £  Rs2  -  R{b.x°[z2(R-l)+R]/(2R)  +  b  x°}2/z2 
1  1  o  11  *■   <■ 

for  0  £  x°  <  (b2x2)/(kbx) 


xplanation  of  Symbols 

(1)  For   t  ,  t   and   t  ,   see  Table  I. 

(2)  k  =  [z2  -  R(z-1)2]/(2R). 
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the  various  types  of  optimal  control,  have  been  deleted  in  constructing 


Case  (a)  of  Table  II  from  Case  (2)  of  Table  I.   For   0  £  6  <  R  -  /R(R-l), 
D(C  ),   the  domain  of  controllability  for  the  C   part  of  the  terminal 
surface,  and  D(C  )   have  a  non-empty  intersection  and  similarly  for 
D(C.)   and   D (C ,) .   Hence,  step  (e)  in  our  above  solution  procedure 
must  be  applied.   In  section  5.3  the  determination  of  the  two  dispersal 
surfaces  is  shown.   The  resulting  solution  is  shown  in  Table  II  as 
Case  (b). 

The  above  solution  procedure  may  be  easily  extended  to  terminal 
control  differential  games  (such  as  [19]  in  which  the  subsequently 
well-known  necessary  conditions  [1]  were  not  applied).   Moreover,  in 
two-sided  problems  (both  the   X-force  and  the  Y-force  are  heteroge 
eous  and  each  is  faced  with  a  fire  distribution  problem  affected  by 
the  other's  choice)  this  author  has  noted  that  not  only  may  domains 
of  controllability  overlap  with  there  being  multiple  extremals  from  a 
given  point  in  the  initial  state  space,  but  also  the  corresponding 
terminal  states  imply  that  the  same  player  may  either  win  or  lose 
depending  upon  which  extremal  path  is  followed.   This  is  illusory, 
however,  since  the  player's  opponent  may  block  his  steering  the  course 
of  battle  to  a  winning  end  point  by  use  of  a  non-extremal  strategy. 
Hence,  the  player  must  always  lose. 

We  should  note  that  in  the  existing  theory  of  differential 
games  [2],  [14]  it  is  assumed  that  the  optimal  strategies  are  pure. 
Since  the  Hamiltonian  is  separable  (a  function  independent  of  the 
X-force 's  strategy  variable(s)  plus  a  function  independent  of  the 
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Y-force's  strategy  variable(s))  for  two-sided  fire  distribution 
problems  in  the  Lanchester  theory  of  combat  as  formulated  by  Isbell 
and  Marlow  [11]  and  Weiss  (see  pp.  94-95  of  [18]),  it  may  be  shown 
that  there  do  exist  pure  strategy  solutions  to  such  problems  [8]. 
Thus,  this  paper's  developments  are  applicable  to  this  class  of 
problems . 

5.   Development  of  Solution. 

The  solution  is  actually  derived  for  a  "reduced"  game  (that 
portion  of  battle  during  which  Y   is  faced  with  a  choice  problem) . 
We  illustrate  here  for  extremals  to   C  .   It  suffices  to  trace  extre- 
mals up  to   t   when  x  (t  )  =  0,   since  <p*   =   0   from  then  until  the 
end  of  the  game.   The  determination  of  the  value,  denoted  by  V(x  ,x  ,y), 
of  the  reduced  game,  which  is  needed  to  determine  the  values  of  the 
adjoint  variables  on  the  terminal  surface,  and  part  of  the  solution 
originally  obtained  by  Isbell  and  Marlow  will  not  be  repeated  here 
although  we  shall  outline  the  general  steps. 

The  Hamiltonian  is 


H(t,x.,p.,0)  =  -{p^y  +  p2(l-0)a2y  +  P3(b1x1  +  b^)},   (2) 


and  the  adjoint  equations  are 

IT  "   blP3' 
dP2   u 


dt    "2^3 

dT  =  W  +  p2(1-*)V  (3) 
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with 


p  (t=t1)  =  unspecified 


p0(t=t.)  = 


2  dxn  it 2 Z 

2    /b2X2  "  a2y 


8V         qa2y 


P,(t=t1)  =  —  =  — ~ (4) 

,2  /b^  "  a2' 


3         3y    /bT  ,/b^  -  a,y2 


The  extremal  control   0*   is  determined  by  the  maximum  prin- 
ciple, i.e.   maximum  H(t  ,x  .  ,p  .  ,0)  ,      and  we  also  have  that 
0*05:1        x  1 

H(t,x.,p.,0*)  =  0.  (5) 

Equations  (4)  and  (5)  may  be  combined  to  yield  that  for  extremals 
leading  to   C.   we  have 


-qa2/b^  x_(t=t   ) 


or 


pl(t=tl    }    =  -*  <6) 

aly/b2X2   ~  a2yZ 

^'"V*  =ijp2(ti)-  (7) 


where 


P1(t=t1  )  =  lim  p  (t). 
t^t 

Obtaining  a  solution  to  this  problem  is  simplified  by  the 
following  considerations.   Let   t  =  t..  -  t   and  define 

v(t)  =  a  p  (t)  -  a  p1(t),  (8) 
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then  we  have 


with 


£-  (a1b1-a2b2)p3(x),  (9) 


v(x=0)  -  a2p2(x=0)  -  alP;L(T=0),  (10) 


and  where  (up  until  the  first  shift  of  tactics) 

P3(t)  =  p3(t=0)  cosh{/0a1b1  +  (I^aTbT  t} 


0alP;L  (x=O)+(l-0) a2p2  (t=0) 
/0a1b1  +  (1-0)  a^ 


sinh{/0a  b  +  (1-0) a  b   t}  .  (11) 


It  is  easily  seen  that   p  (t)  >  0   for  all   x,   and  thus 

^-   >  0   for  all   t,  (12) 

dx 

when  a  b   >  a  b   by  (9).   As  determined  by  the  maximim  principle  and 
use  of  (2)  and  (8) ,  the  extremal  control  is  given  by 

1   for  v(x)  >  0 

0   for  v(x)  <  0.  (13) 

It  is  easy  to  show  that  it  is  impossible  for  v(x)  =  0   over  any  finite 
interval  of  time,  and  hence  the  possibility  for  any  singular  solution 
[15]  to  this  problem  is  excluded.   (The  reader  should  note  that  the 
term  "singular  surface"  is  used  in  a  slightly  different  sense  by 
Isaacs  (see  pp.  132-134  and  pp.  156-158  of  [14])  and  in  the  control 
theory  literature  (see  p.  502  of  [10]  or  p.  4  of  [15]).) 
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By  the  symmetry  of  the  problem  it  suffices  to  assume  that 
a  b   >  a?b_,   and  then  equations  (7),  (8),  (12)  and  (13)  imply  that 
extremals  can  actually  lead  to  C  .   Furthermore,  for   a  b   >  a_b 
it  is  readily  shown  that  no  extremals  lead  to  C    (i.e.   D (C  )   is 
void)  and  that  extremals  can  only  lead  to  C,   when  x  (t)  >  0   for 
t  <  T.   We  shall  sketch  the  proof  that  no  extremals  for  which  x_(t)  =  0 
for   t  <  T  can  lead  to  C , .   By  arguments  similar  to  those  used  above 
for  C   and  using  the  value  of  the  reduced  game  V(x  ,x9,y)   and  (2) 
and  (5)  with   t  =  0  and   0*  =  0 ,   it  is  readily  shown  that  v(t=0  )  =  0 
where   t   is  the  backwards  time  from  the  end  of  the  reduce  game,  i.e. 
x„(t=0)  =  0,   and  v(i=0  )   is  a  one-sided  limit  through  positive 
values.   Considering  (12),  this  implies  that  v(t)  >  0   for  t  >  0, 
and  hence  by  (13)   X   is  never  fired  at  so  that   C,   cannot  be  reached. 
The  special  case  in  which  C,   is  reached  but  x  (t)  >  0   for   t  <  T 
is  discussed  in  section  5.2. 

In  the  next  three  subsections  we  present  additional  details 
of  solution  development:   (1)  determination  of  domains  of  controlla- 
bility, (2)  entry  to  C  ,   and  (3)  determination  of  dispersal  surfaces. 
This  material  represents  the  extension  of  the  original  work  of  Isbell 
and  Marlow  [11] . 


5.1.   Determination  of  Domains  of  Controllability. 

One  contribution  of  this  note  is  to  show  how  to  determine  the 
domains  of  controllability.   There  are  two  cases  to  consider. 
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Case  (1)   a  q  £  a  p 

These  determinations  are  routine  and  consist  in  combining  the 
time  history  of  the  extremal  control,  the  non-negativity  requirements 
of  the  state  variables,  and  the  generalized  square  law 

u2(tl)  -  u2(t2)  =  {0a1b1  +  (l-0)a2b2}(y2(tl)  -  y2(t2>),    (14) 

where  0(t)  =  const.   in   t.  £  t  £  t„  and  u(t)  =  b  x  (t)  +  b  x  (t) . 

Case  (2)   a_q  >  a  p 

The  condition  for  entry  to  C        is  as  before.   There  are  two 
subcases  for  entry  to  C  .   First  we  consider  the  extremals  for  which 


0*(t)  = 


for  0  £  t  £  T  -  t 


0   for  T  -  t-   £  t  £  T.  (15) 


Combining  the  above  extremal  control  with  the  generalized  square  law 
(14) ,  we  obtain 

R(b2x2(T))2  +  2Rb1b2x1(T-T1)x2(T)  +  (b^CT-T^)  2 

+  2(l-R)b1b2x1(T-T1)x°  +  a^y^  -  s2  +  (l-RMb^)2  =  0.   (16) 

A  backwards  integration  of  the  state  equations  of   0  £  t  £  T-,   with 
x  (T=0)  =  x  (T)   and  y(i=0)  =  0  yields  that 

b2X2  =  b2X2(T)z  +  b1x1(T-T1)(z"1)»  <17> 

since   x  (t=T-i  )  =  x  .   It  should  be  noted  that  in  deriving  (17)  we 
have  assumed  that   x  (T-t  )  >  0.   (16)  and  (17)  may  be  combined  to  yield 
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(b2x2(T)-b2x°)2  =  (aibiyo2-s2)/A,  (18) 

and  hence  we  require  that  the  right-hand  side  of  (18)  be  non-negative 
in  order  that  C   be  reached.   It  may  be  shown  that  A  >  0   leads  to 
a  contradiction  to  a  result  (32)  derived  below,  and  hence  A  a£  0   for 
extremals  corresponding  to  (15)  to  lead  to  C  .   This  corresponds  to 
R  -  /R(R-l)  <  6  <  1,   since  A  =  [z2(R-l)  -  R]/z2   and   z  =  (R-6)/(R-l) 
When  A  ri.   0 ,   solution  of  (18)  for  x  (T)   and  the  requirement  that 
x  (T)  >  0   leads  to 

a  by  2  >  s2  +  A(b.xV.  (19) 

1  1  o  2  2 

Combination  of  the  expression  for  x  (T)   with  (17)  and  the  require- 
ment that   x  (T-t  )  >  0   leads  to 

alblyo2  <  s2  +  (b2x2)2A(z~1)2/z2-  (20) 

Next,  we  consider  the  extremals  for  which 

<p*(t)    =   0   for   0  £  t  *  T.  (21) 

It  is  readily  shown  that 

(b1x1  +  t>2x2) 
y(t)    =  y     cosh/a  b      t -—^ sinh/a  b      t,  (22) 

so  that  the  boundary,  which  occurs  when  y(T)  =  0   for  T  =  t  , 
between  this  case  and  the  previous  one  is  easily  seen  to  be  given  by 


* 
There  is  a  flaw  in  this  argument.   The  final  solution,  however,  is  not 

affected  by  this.   See  Appendix  F  for  a  further  elaboration  of  this  and 

other  subtle  points. 
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, (b   x°+b   x°)2 

y2  [cosh/a  b      t    ]2   =  K  {[cosh/aTbT  tJ2   -  1},      (23) 

o  2   2      1  a?b9  221 

where  the   T-time  of  the  switch,  determined  from  integration  of  the 
adjoint  equations  and  the  maximum  principle,  is  given  by 


__      ai   Cb  q  -  b2p) 

z  =  cosh /a  b  T     ■  —  - — - — — -  .  (24) 

2  2   1    q  (a^b   -  a2b2) 


Noting  that   0*  =  0   for  the  entire  battle  when  T  <  x   and  re-arrang- 
ing, we  obtain  the  result  shown  in  Table  I.   Combination  of  the  extremal 
control  (21)  with  (14)  also  yields  a  quadratic  equation  for  b  x  (T) 
which  may  be  solved  to  yield 

b2x2(T)  =  -bxx°  +  /sz  -  a2b2yQ^.  (25) 

Thus,  the  payoff  for  extremals  on  which  (21)  holds  is  given  by 

P5  "  £  '  (Vl>  -^T1  -  ^  -  ^V7)75}.         (26) 

The  requirement  that  x_(T)  >  0   leads  to  the  result  shown  in  Table  I. 
When  this  latter  condition  or  (19)  is  violated,  then  it  is  not  possible 
for  x?(T)  >  0  when  0*(t)  =  0   for   0  i  x  £  t, .   Hence,  we  must 
investigate  the  possibility  that  x  (T)  =  0   (C,   is  entered),  and 
this  is  done  in  the  next  section. 

Finally,  we  consider  extremals  leading  to  C  .   As  done  above, 
we  may  combine  the  extremal  control  with  (14)  and  the  requirement  that 
x  (T)  >  0   to  obtain  the  first  result  shown  in  Table  I.   Similarly, 
y(t-)  ^  0   implies  that 
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aiblYo2  *  (blXl)2  +  2blVlV  (27) 

Also,    the   payoff    for   extremals    leading    to      C        is   given  by 

P,  .= ±~  Ss*  +    (R-l)(b_x;)^   -  a  by   2",  (28) 

1  ^^  2    2  1    1   o 

It  remains  to  develop  conditions  which  distinguish  between 
optimal  paths  leading  to   C   and  C  .   (Thus,  strictly  speaking  the 
result  for   C    shown  in  Table  I  Case  (2)  is  for  the  region  where 
extremals  of  D(C  )   yield  a  larger  payoff  than  extremals  to   C   when 
the  two  fields  of  extremals  overlap.)   We  do  this  for  A  <  0,   i.e. 
z2  <  R/(R-1),   since  different  arguments  are  required  to  determine 
optimal  trajectories  when  A  >  0   (see  section  5.3).   It  will  be  shown 
that  optimal  paths  can  only  lead  to   C.   when  the  time  remaining  in  the 
battle  after  destruction  of  the   X    forces  exceeds  the  maximum  time 
to  fire  at   X_   for  extremals  reaching  C  . 

Let   t   =  T  -  t   denote  the  length  of  time  that  Y  fires  at 

X   after  X   has  been  annihilated.   Then  optimal  paths  can  only 

reach   C   when   T   ^  x  (C  )  =  t  ,   where   t  (C  )   is  the  "backwards 

time"  of  the  switch  in  tactics  for  C_   extremals.   First,  we  shall 

show  that  extremals  from  P   with  x   >  0   leading  to  C   yield  a 

better  payoff  than  those  leading  to   C   for  a-,b-iy  2  *  Rs2{l  -  1/z2} 

and   z   <  R/(R-1).   Thus,  optimal  paths  can  only  lead  to   C.   from 

initial  points  for  which  a.,b,y  2  >  Rs2{l  -  1/z2}.   However,  for  such 

1  1  o 

points  the  extremal  strategy  is  to  use   0*(t=O)  =  1   and  to  continue 
this  until  either  the  switching  surface  is  encountered  or  X   is 
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annihilated,  i.e.   0*(t)  =  1   for   0  ■£.   t  £  minimum  (T-t  ,  T-x  )  . 

Hence,  optimal  paths  can  only  reach  C   when  t   ^  t  . 

Thus,  it  remains  to  sketch  the  proof  that  extremals  from  P 

with  x  >  0   leading  to  C   yield  a  larger  payoff  than  those  leading 

to  C.   for  a.b.y  2  <  Rs2{l  -  1/z2}   and   z2  <  R/(R-1).   Setting 
1        1  I  o 

a.b.y  2  =  Rs2{l  -  1/z2}   in  the  expressions  for  Pr   and  P,   (equa- 
1  1  o  5        1 

tions  (26)  and  (28),  respectively),  it  is  easily  shown  that  P_  >  P 

3P1   8P5 

for   z2  <  R/(R-1).   Then  since  - —  S: ^  0,   it  is  readily  seen 

3y    3y  J 

o    J  o 

that  P_  >  P,   for  all  y   such  that  anb,y  2  £  Rs2{l  -  1/z2}   when 
j    1  o  1  1  o 


A  <  0. 


that 


Combining  the  extremal  control  for  C   with  (14) ,  we  have 


y2^   '  4l  UlblyoZ   "   (blXl>2  "  2blb2XlX2K  (29) 

Also,  since  0*(t)  =  0   for   t]  £  t  £  T,   integration  of  the  state 
equations  (1)  yields  that 

y(t)    =  y(t=t1)cosh/a2b2    Ct-^)    -  ^/a     sinh/a2b2    (t_tl)  *    (30) 


Since  y(t=T)  =0,   we  have  that 


y(t=t1)  /a. 


tanh/a^  Tj  =  —^ /  — .  (31) 

Since  the  hyperbolic  cosine  is  a  strictly  increasing  function  of  its 


argument,   x   ^  x    implies  that   cosh/a  b   t..  ^  cosh/a  b   T-  £  1. 

Combination  of  this  with  (29)  and  (31),  use  of  well-known  identities 

for  the  hyperbolic  functions,  and  some  algebraic  manipulation  yields  that 
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3^2  *  (blXJ+b2X°)2  +  {''<8ff  -  R]  (b2x°)2.       (32) 

For   1  £  z2  £  R/(R-1),   it  is  easily  seen  that  (32)  implies  (27)  so 
that  this  latter  condition  need  not  be  considered  further. 


5.2.   Entry  to  Terminal  State  C, . 

Terminal  state  C.   is  reached  whenever  X„   can  be  annihi- 
4  2 

lated  by  Y  during  the  battle's  last  phase  of  length  t. .   The  bound- 
ary conditions  on  the  terminal  surface  are 


Xl(T)  >  0     p1(t=0)  -  -p 

x  (T)  =  0     po(x=0)  =  p_,  unspecified 

y(T)   -  0     p  (t=0)  =  p°  unspecified  (33) 

We  also  have  that  0*(t=T)  =  0*(t=O)  =  0,   and  hence  by  (33)  and  the 
transversality  condition  (5)  it  follows  that   p  =  0.   It  should  be 
noted  that   p   remains  unspecified  on  the  terminal  surface.   It  will 
be  shown  that   p   is  determined  by  conditions  for  extremals  to  reach 
C,.   Recalling  the  extremal  control  at   t  =  T   and  (13),  we  also  must 
have  that 

alP/a2  <  (-p°).  (34) 

Since  v(t=0)  <  0  and  -—  >  0  by  (12),  there  will  be  a 

dx 

switch  in  tactics  in  "backwards  time"  at   t  =  -^(C.)  =  t,  .   (In  this 

14     1 

section  the  notation  t,   refers  to  extremals  leading  to  C.   when  it 

1  4 

is  not  denoted  otherwise.)   Thus,  the  extremal  control  is  given  by 
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0*(t)  = 


for  0  £  t  £  t_ 


_0   for   t  *  t  £  T,  (35) 


where   t  =  T  -  x   and   t   is  such  that 

2blX;L(t2)x°  +  b2(x°)2  =  a2y2(t2).  (36) 

From  a  backwards  integration  of  the  adjoint  equations  and  the  maximum 
principle,  the  time  of  the  switch  in  tactics  is  given  by 

al  (blP2  +  b2P) 


w  =  cosh^  x±   -  -^  (aibi  _  a^^  -  w(p2)  .        (37) 


We  observe  that  from  what  region  of  the  initial  state  space  extremals 
lead  to   C,   is  determined  by  the  value  assigned  to   p_.   In  other 
words,  the  value  assigned  to  p_   determines  what  initial  force  levels 
x   are  to  be  totally  destroyed.   It  seems  reasonable  to  argue  that 
(-p9)   cannot  exceed   q,   since  otherwise  we  would  implicitly  be  valuing 
X„   survivors  greater  than  stated  for  the  problem.   A  more  precise 
argument,  of  course,  is  to  compare  payoffs  (i.e.   P.   and  P  ,   as 
done  in  section  5.1)  for  extremals  leading  to  C.   and  Cr   from  the 
same  initial  point  P   corresponding  to  p   >  q,   but  this  has  not 
proven  to  be  computationally  tractable.   Thus,  recalling  (34),  we  must 
have 

a1p/a2  <  (-p°)  si  q.  (38) 
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As  done  previously,  we  combine  the  extremal  control  (35)  with 
(14)  to  obtain 

(blXl(t2))2  -  2(R-l)b2x°b1x1(t2)  +  a^y^  -  s2  -  (R-lMb^)2  =  0,    (39) 

which  may  be  solved  for   x..  (t„)   to  yield 

b1x1(t2)  =  b2x°(R-l)  +  /sz  +  R(R-l)(b2xpz  -  a^y^,     (40) 

whence  the  result  shown  in  Table  I  by  requiring  that  the  quantity  under 
the  radical  sign  is  non-negative.   The  other  entries  for  C,   in  Table 
I  correspond  to  other  parts  of  the  terminal  surface  being  impossible 
to  reach. 

It  remains  to  show  that  optimal  paths  can  only  lead  to   C. 


from  P   when 


wh 


a^^2  s:  s2  +  A(b2x°)2,  (41) 

ere  A  =  A(R,z)   and   z  =  cosh/aTbT"  i-CC^).   A  backwards  integration 

of  the  state  euqations  (1)  for   0  £  t  £  t,  (C.)  =  t,   with  xn  (t=0)  = 

14     11 

x  (t  )   and  x  (t=0)  =  0  yields  that 

b2X2 

Xl(t2}  =b7(Mr  (42) 

since  x„(t=t1)  =  x   and  x  (t=t  )  =  x  (t»).   Combining  (39)  and  (42), 
we  obtain 


s2  +  A(R,w)(b2x°)2  -  a^^2  =  0.  (43) 
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For  constant   R,   A  may  be  considered  to  be  a  function  of   w.   Con- 
sidering the  first  two  derivatives  of  A(w)   and  lim  A(w) ,   it  is 

W-+00 

readily  seen  that  for   1  jS  w  sl   +°°,   A(w)   has  a  global  maximum  at 
w*  =  R/(R-1)   with  A(w*)  =  R(R-l)  >  0.   This  latter  result  implies 
that  for  P   satisfying  (41)  the  quantity  under  the  radical  sign  in 
(40)  is  non-negative  (i.e.  the  corresponding  entry  in  Table  I  is 
satisfied) .   It  should  be  observed  that  w(-p  )   is  a  strictly  in- 
creasing function  of   (_P9)   with 


w 


"Po  = 


alp 


2    a2 


=  1   and  w(-p2=q)  =  z  =  ^-  £  ^— j-.       (44) 


Since  for   1  £  w  £  w* ,   A(w)   is  a  strictly  increasing  function;  then 

a  p 

for  iQ.    (-p9)  iC   q,   A(w(-p  ))  is  a  strictly  increasing  function 

a„       z  L 

whence  (41).   Also,  combination  of  (37),  (40)  and  (42)  yields  that 

a.p{b,x°(R-l)  +  /S(PU,R)} 

(-P°(P°))=— l   \       r        (45) 

a2/S(P°,R) 

where 

S(P°,R)  =  s2  +  R(R-l)(b2x°)2  -  a^y^.  (46) 

Equation  (45)  shows  the  dependence  of  dual  variable  values  on 

the  terminal  surface  upon  the  initial  force  levels  P   of  an  extremal 

It  seems  appropriate  to  observe  that  for  a  p/a  £  (~P9(P  ) )  ^  q  we 

have  that   0  £  T,  (C.)  £  x,  (Cr)   and  also   t-.(C.)  =  x,  (P°)  .   The  value 
1   4      1   j  14      1 

implicitly  assigned  to   X   survivors  through   p    is  adjusted  so  that 
the  X    forces  are  just  annihilated  at   t  =  T.   Let  us  also  note  that 
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by  construction  we  have  made  v(x=x  (P  ))  =  0   in  our  determination 
of   (-p0(P  )).   The  above  should  make  clear  the  reason  that   p   must 
be  unspecified  on  C . :   for  extremals  corresponding  to  a  given  value 
of  x1 (T)   and  leading  to  C,   there  are  an  infinite  number  of  different 
values  for  x,  (P  )   for  these  extremals  that  result  in  this   x  (T) . 


5.3.   Determination  of  Dispersal  Surfaces. 

It  seems  appropriate  to  note  here  some  relationships  between 
the  inequalities  which  define  the  domains  of  controllability  in  Table 
I.   It  is  always  true  that 

s2  +  B(b2x°)2  s:  s2  +  (R-l)(b2x°)2.  (47) 

For   R  -  /R(R-l)  £  6  <  1,   the  following  hold 

s2  +  A(b2x°)2  £  s2  +  (R-l)(b2x2)2,  (48) 

Rs2{l  -  1/z2}  *  s2  +  (R-l)(b2x°)2,  (49) 

s2  +  A(b2x°)2  s  s2  +  B(b2x°)2,  (50) 

Rs2{l  -  1/z2}  £  s2  +  B(b2x°)2.  (51) 

In  (51),  there  is  strict  inequality  for  x  >  0  and  R  -  /R(R-l)  <  6  <  1 . 


For  0  n.   6  <  R  -  /R(R-l)  ,   we  also  have 


s2  +  A(b2x°)2  >  s2  +  B(b2x2)2,  (52) 


Rs2{l  -  1/z2}  >  s2  +  B(b2x°)2.  (53) 
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The  proofs  of  the  above  are  straight-forward  and  omitted. 

When   0  £  6  <  R  -  /R(R-l) ,   the  above  inequalities  (52)  and 

(53)  imply  that  D(C  )   and  D(C  )   for  extremals  on  which  <p* (t)    =  0 

for  0  i  t  iT  overlap  (i.e.  there  is  a  region  in  the  initial  state 

space  from  which  extremals  lead  to  both  C   and  C  )   and  similarly 

for  D(C.)   and  D(C.)   when   a.b.y  2  £  s2  +  A(bnx°)2   in  D(C,). 
1  4  llo  22  4 

Hence,  we  must  consider  step  (e)  of  section  4.'s  solution  procedure. 
First,  we  consider  the  determination  of  the  dispersal  surface 
which  separates  regions  of  the  initial  state  space  from  which  optimal 
paths  lead  to   C    and   C,..   Let  us  observe  that  for   z2  >  R/ (R-l) 
points  exist  for  which 

s2  +  B(b2x°)2  <  a  b  y  2  £  Ks2{l  -  1/z2}.         (54) 

Setting  a-ib,y  2  =  s2  +  B(b  x  )2   in  the  expressions  (26)  and  (28)  for 
P   and  P   respectively,   it  is  easily  shown  that  it  is  optimal  to 
go  to   C   from  such  an  initial  point.   The  dispersal  surface  is  the 
locus  of  points  in  the  initial  state  space  for  whicb   P   equals  P 
and  extremals  lead  to  both   C    and   C    on  both  sides  of  this  sur- 
face.  Equating  (26)  and  (28),  the  locus  of  points  for  which  P   =  P- 
is  given  by 

a-t^y  2  =  Rs2  -  R{bx°[z2(R-l)  +  R]/(2R)  +  b0x°}2/z2.     (55) 
llo  1  i  l   i 

Furthermore,  a  dispersal  surface  actually  exists,  since  it  is 
readily  shown  that 
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Rs2  -  R{b  x°[z2(R-l)  +  R]/(2R)  +  b2x°}2/z2  <   Rs2{l  -  1/z2},   (56) 

when  z2  >  R/(R-1).   (56)  and  an  above  result  mean  that  on  both  sides 

of  the  candidate  dispersal  surface  extremals  lead  to  both  C   and  C  . 

dP±        3P5 

Observing  that  - —  ^  - —  ^  0,   it  is  readily  seen  that  Pn  ^  PP   for 
3yo   9yQ  1    5 

all  P°   such  that   a  by   ^  Rs2  -  R{b  x  [z2(R-l)  +  R]/(2R)  +  b  x°}2/z2 
Since  we  also  must  have  a^^  2  <  R{s2  -  (b-.x..)2}   for  extremals 
leading  to  C,.,   the  above  dispersal  surface  only  exists  for  0  i  x..  < 
2Rb2x°/{b1[z2  -  R(z-l)2]}. 

Based  on  the  above  results  there  may  also  be  a  dispersal  sur- 
face which  separates  optimal  paths  leading  to  C   and  C,   for 
0  •£  x  <  b  x  [z2  -  R(z-l)2]  /  (2Rb  ).   Its  determination  is  similar  to 

the  above,   although  the  details  are  messier  and  haven't  been  completely 
worked  out.   Let 

alblyo   =  g(p°>R>z)>  (57) 

be  the  locus  of  points  for  which   P   =  P  .   Then,  the  boundary,  which 
separates  optimal  paths  leading  to   C,   and  C , ,   is  given  by 


alVo   =  h(p°>R>z)>  (58) 


where 


s2  +  B(b2x2)2   for   g(Pu,R,z)  <  s2  +  B(b2x2)2 

(P°,R,z)    {   g(P°,R,z)       for   s2  +  B(b2x°)2  is.   g(P°,R,z)  i  s2  +  A(b2x°)2 

s2  +  A(b2x°)2   for  g(P°,R,z)  >  s2  +  A(b2x°)2.  (59) 

It  should  be  noted  that  this  boundary  surface  is  a  dispersal  surface 

only  when  extremals  to  both   C,   and   C.   lie  on  both  sides  of  a,b.,y2 

14  1  1  o 

=  g(P°,R,z). 
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6.   Structure  of  the  Optimal  Allocation  Policies. 

For  square-law  attrition  it  may  be  shown  that  the  allocation 
of  fraction  of  fire  is  always  0  or   1.   In  section  5  we  have  dis- 
cussed the  fact  that  the  coefficient  of  the  control  variable  <p     in 
the  Hamiltonian  H(t,x.,p.,0)   cannot  be  equal  to  zero  over  any  finite 
interval  of  time.   This  excludes  the  possibility  of  a  singular  control 
[15],  and  maximization  of  the  Hamiltonian,  which  is  a  linear  function 
of  <p,      yields  that  all  fire  is  concentrated  on  one  target  type.   This 
is  not  surprising,  since  our  model  assumes  complete  and  instantaneous 
information  [2]  and  that  fire  may  be  immediately  shifted  to  a  new 
target  once  the  old  one  has  been  destroyed  [5],  [18]. 

With  reference  to  Tables  I  and  II,  the  condition  that   a  b   > 
a„b   may  be  interpreted  to  mean  that  there  is  more  long  range  return 
for  Y   to  engage  X..,   i.e.  more  Y's  will  survive  if  this  is  done. 
Hence,  when  Y  wins,  he  always  engages   X  's  while  they  are  available. 
The  condition  a  p  <  a  q  means  that  at  the  end  of  battle  there  is 
greater  payoff  per  unit  time  per  Y   soldier  to  engage  X„   not  con- 
sidering X  's  greater  attrition  effect  against  Y   (short  term  gain 
at  end  of  battle) . 

By  the  maximum  principle  and  the  well-known  interpretation  of 
the  dual  variables  [1],   Y   always  allocates  his  fire  entirely  to  the 
target  type  yielding  the  greatest  marginal  return.   However,  marginal 
return  evolves  differently  in  winning  or  losing  causes.   When  Y  loses, 
he  may  switch  from  firing  at  X   entirely  to  firing  at   X   entirely 
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to  firing  at  X_   entirely  before  the  X   force  has  been  annihilated. 
This  happens  when  Y   assigns  utility  to  survivors  of  force  type  X 
in  excess  of  their  kill  rate  against  Y   as  compared  to  force  type 
X  ,   and  X   is  abundant  enough  not  to  be  destroyed  before  the  battle 
ends.   Under  these  circumstances,  whenever  the  X„   forces  are  annihi- 
lated, this  occurs  at  the  very  end  of  battle. 

In  this  way,  we  see  that  tactics  may  depend  on  force  levels. 
Moreover,  the  time  at  which  the  ranking  of  target  priorities  switches 
itself  may  depend  upon  force  levels.   We  also  see  that  Y's   target 
priorities  only  switch  with  time  in  a  losing  case.   This  has  occurred 
since  a  boundary  condition  at   t  =  T  on  one  of  the  dual  variables  is 
dependent  upon  values  of  che  state  variables  by  a  transversality  con- 
dition.  It  may  be  shown  that  the  structure  of  optimal  allocation 
policies  is  different  for  the  prescribed  duration  battle. 
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Appendix  B.   Lanchester-Type  Models  of  Warfare  and  Optimal  Control. 

1 .   Introduction. 

The  optimization  of  the  dynamics  of  combat  is  studied 
through  a  sequence  of  idealized  models  by  use  of  mathematical  theory 
of  optimal  control.   The  models  are  for  combat  over  a  period  of 
time  described  by  Lanchester-type  equations  with  a  choice  of  tactics 
available  to  one  side  and  subject  to  change  with  time.   A  sequence 
of  models  is  examined  for  each  of  two  types  of  choice  situation: 

selection  of  target  type  for  engagement, 
regulation  of  firing  rate. 

Optimal  tactics  are  discussed  with  reference  to  the  influence  of 
combatant  objectives,  termination  conditions  of  conflict,  type  of 
attrition  process,  variable  attrition  rates,  and  limited  ammunition. 
Implications  for  intelligence,  command  and  control  systems,  and 
human  decision  making  are  pointed  out.   The  use  of  such  optimal 
control  models  for  guiding  extensions  to  differential  games  is 
discussed. 
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In  this  appendix  we  examine  the  structure  of  optimal  allocation 
policies  for  tactical  situations  describable  by  Lanchester-type 
equations  of  warfare.   We  hope  to  provide  insight  into  such  questions 
as 

(1)  How  should  targets  be  selected? 

(2)  Do  target  priorities  change  with  time? 

(3)  Does  the  number  of  target  types  effect  the  selection? 

(4)  Do  battle  termination  circumstances  effect  the  op  'onal 
allocation  policies? 

(5)  How  does  the  nature  of  the  attrition  process  effect 
target  selection? 

(6)  What  is  the  effect  of  ammunition  constraints? 

(7)  How  does  the  uncertainty  and  confusion  of  combat  effect 
the  optimal  selection  rules? 

Our  theory  of  tactical  allocation  is  developed  through  the  examination 
of  a  sequence  of  simplified  models.   These  combat  models  are  too  simple 
to  be  taken  literally  but  should  be  interpreted  as  indicating  general 
principles  to  serve  as  hypotheses  for  subsequent  computer  simulation 
studies  or  field  experimentation. 
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In  1964  Dolansky  [10]  noted  that  the  Lanchester  theory  of 
combat  was  insufficiently  developed  in  the  area  of  target  selection 
for  combat  between  heterogeneous  forces  (optimal  control/differential 
games).   Even  the  two  references  cited  by  him,  Weiss  [32]  and  Isbell 
and  Marlow  [ 16 ] ,  have  been  subsequently  extended  by  this  author  [26], 
[28].   Since  Dolansky's  article,  no  further  examples  have  been  published 
in  the  literature  except  for  the  ones  in  Isaacs'  book  [15].   This  pre- 
vious work  had  never  systematically  investigated  the  dependence  of 
tactics  upon  model  form. 

We  examine  several  idealized  combat  situations  described  by 
Lanchester-type  equations  over  a  period  of  time  with  choices  of  tactics 
available  to  one  side  and  subject  to  change  with  time.   We  consider 
models  for  two  types  of  choice  situations 

(1)  selection  of  target  type, 

(2)  regulation  of  firing  rate. 

These  problems  are  solved  by  the  mathematical  theory  of  optimal  control. 
A  further  elaboration  on  solution  development  is  to  be  found  in  our 
report  [25]. 

In  the  first  sequence  of  models  we  examine  the  effect  on 
optimal  target  selection  of  the  following  factors:   objectives  of  the 
combatants,  termination  conditions  of  the  conflict,  number  of  target 
types,  some  special  cases   of  time  dependent  attrition  rates,  and  type 
of  attrition  process.   We  then  examine  a  sequence  of  models  to  see  how 
ammunition  limitations  effect  firing  rates.   Next  we  discuss  two-sided 
extensions  of  such  problems  but  point  out  the  value  of  studying  one- 
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sided  problems  as  considered  in  this  appendix.   Finally,  various 
implications  of  the  models  are  discussed. 

2.   Target  Selection. 

We  begin  by  considering  the  simplest  situation  of  target 
selection:   combat  between  an  X-force  of  two  force  types  (for  example, 
riflemen  and  grenadiers)  and  a  homogeneous  Y-force   (for  example, 
riflemen  only) .   This  situation  is  shown  diagrammatically  below. 


It  is  the  objective  of  the  Y-force   commander  to  maximize  his  survi- 
vors at  the  end  of  battle  at  time  T   and  minimize  those  of  his 
opponent  (considering  weighting  factors   p,   q  and   r) .   This  is 
accomplished  through  his  choice  of  the  fraction  of  fire,   0,   directed 
at  X  .   We  may  study  this  idealized  tactical  allocation  problem  in 
two  types  of  scenarios:   (1)   a  battle  lasting  a  specified  time,   T 
or  (2)  a  battle  lasting  until  one  side  or  the  other  is  totally 
annihilated. 
2.1.   Battle  of  Prescribed  Duration. 

Mathematically  the  problem  may  be  stated  as 
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maximize  ry(T)  -  px  (T)  -  qx  (T)   with  T   specified 
(t)  l  l 

dxl 
subject  to:    - —  =  -  0  a  y 

dx 

"  =  -  (l-0)aoy 


dt      v   1K/  2' 


ft     =  -  b^  -  b2x2 


i.  ,x  ,y  ^  0  and  0  £  0  £  1, 


where 

p,q,   and   r  are  weighting  factors  assigned  to  surviving  forces, 

x i ,x_   and  y  are  average  force  strengths, 

a  ,a  ,b   and  b   are  constant  attrition  rates, 
and   0  is  fraction  of  Y-f ire  directed  at  X- . 

This  problem  may  be  solved  by  routine  application  of  Pontryagin 
maximum  principle*  [22].   The  solution  is  shown  in  Table  I.   In  this 
present  analysis  we  have  not  considered  those  subcases  when  a  state 
variable  is  reduced  to  zero.   It  may  be  shown  that  the  structure  of 
the  optimal  allocation  policies  is  not  fundamentally  altered  by  such 
an  occurrence. 

With  reference  to  Table  I,  we  see  that  two  characteristics  of 
the  optimal  allocation  policies  for  this  particular  prescribed  dura- 
tion battle  are: 

(1)  concentration  of  all  fire  on  one  target  type, 

(2)  independence  of  allocation  from  force  levels. 


ju 

We  employ  the  equivalent  version  commonly  used  in  this  country 
(see  p.  108  of  [9]). 
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We  shall  later  see  that  when  there  are  more  than  two  target  types  in 
this  scenario,  the  solution  possesses  these  same  characteristics  (even 
when  the  attrition  rates  change  over  time) .   Both  these  characteris- 
tics, however,  are  consequences  of  the  assumed  model  form. 

The  first  characteristic,  concentration  of  effort  on  one 
alternative,  is  a  consequence  of  the  "square-law"  attrition  process 
for  the  X-forces.   (We  shall  refer  to  attrition  as  being  a  "square-law" 
process  when  the  casualty  rate  is  proportional  to  the  number  of  enemy  firers 
and  as  being  a  "linear-law"  process  when  it  is  proportional  to  the 
product  of  the  number  of  enemy  firers  and  remaining  targets.)   It  may 
be  shown  that  this  makes  the  existence  of  a  singular  control  [18] 
impossible,  and  hence  the  optimal  allocation  policies  are  extreme 
points  in  the  control  variable  space. 

There  is,  however,  a  very  simple  principle  which  underlies  the 
above  mathematical  formalities:   concentration  of  effort  when  constant 
marginal  returns  are  obtained  from  the  alternatives  and  the  total  effort 
is  limited.   Constant  marginal  effect  over  time  per  unit  of  weapon  system 
is  a  property  of  the  "square-law"  attrition  process,  for  let  us  consider 
the  X  -force  attrition  (when  0=1) 

dxi 


dt 


[rate  of  casualties  produced  per 
y      al    [unit  of  Y-force  weapon  system 

Thus  there  is  a  constant  (or  non-diminishing)  marginal  effect  over 
time.   This  should  be  contrasted  with  the  situation  for  a  "linear-law" 
attrition  of  the  X-forces 
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dt 


=  a 


1*1 


rate  of  casualties  produced  per 
unit  of  Y-force  weapon  system 


In  this  case  we  have  diminishing  effects  over  time  from  allocating  a 
unit  of  Y-force  weapon  system  against   X  ,   and  a  division  of  total 
effort  (i.e.,  fraction  of  fire)  may  be  called  for.   B.  Koopman's  1953 
article  [21]  contains  an  excellent  discussion  of  such  principles 
which  underlie  such  an  optimization  problem.   Presently,  we  shall 
verify  these  heuristic  arguments  in  a  mathematically  precise  fashion 
when  we  consider  a  dynamic  model,  which  considers  the  interaction  of 
forces  over  time,  in  which  both  X-force  target  types  undergo  "linear- 
law"  attrition.   This  fundamental  difference  in  the  structure  of  opti- 
mal allocation  policies  based  on  the  nature  of  target  attrition  makes 
the  determination  of  the  appropriate  attrition  process  an  essential 
task  of  analysis. 

The  second  characteristic,  target  selection  independent  of 
force  levels,  is  due  to  the  combination  of  the  "square-law"  attrition 
process  for  the  X-force   types  and  the  fixed  battle  length,   T.   It 
is  seen  that  for  the  battle  of  prescribed  duration  target  selection 
depends  only  on  the  attrition  rates  of  the  various  force  types  and 
relative  weights  assigned  to  surviving  force  types.   This  is  not  surprising* 
since  it  is  easily  seen  that  the  adjoint  differential  equations  are  independet 
of  the  state  variables,  and  the  values  of  the  dual  variables  at  the 
end  of  battle   t  =  T   are  independent  of  force  strengths.   It  is 
recalled  that  a  dual  variable  represents  the  rate  of  change  of  the 
payoff  (battle  outcome  as  measured  by  the  value  of  surviving  forces 
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at   t  =  T)   with  respect  to  a  particular  state  variable  [3].   Thus, 
if  if   V  =  ry(T)  -  pX;L(T)  -  qx2(T),   then  p^t)  =  ~   (t)  ,   etc. 
Hence,  the  boundary  conditions  are  given  for  the  dual  variables  at 
the  end  of  the  battle   t  =  T   as   p  (t=T)  =  —  (t=T)  =  -p,p  (t=T)  = 

J-  'J  X  _  ^ 

-q,P3(t=T)  =  r. 

It  seems  appropriate  to  discuss  further  the  interpretation  of 
the  solution  shown  in  Table  I.  From  the  above  definition  of  the  dual 
variables , 


a1P1(t)  = 


effect  on  outcome  per 


kill  rate  of 


unit  time  for  engaging  X  J  [Y  against  X  J  (unit  of  X   destroyed^ 


effect  on  outcome  per 


Hence,  the  condition  a.   p  <  a„q  means  that  at  the  end  of  the  battle 
(recall  that   p  (t=T)  =  -p,  etc.)   there  is  greater  effect  on  battle 
outcome  (as  measured  by  value  of  survivors)  per  unit  time  per  soldier 
for  Y   to  engage  X    (short  term  gain  at  the  end  of  battle) .   The 
value  of  the  dual  variable,  for  example,   P-,  (t)   reflects  both  the 
value  assigned  X  -force   survivors  and  the  dynamic  interaction  of 
forces  over  time  through  the  Lanchester-type  equations.   Hence,  it 
also  accounts  for  the  effectiveness  of  X   against  Y.   The  quantity 
a..b   may  be  interpreted  as  representing  the  instantaneous  rate  of 
destruction  of  the  X  -force  kill  rate  against  the  Y-force  per  unit 
of  Y-force.   Then  a  b   >  a  b   means  that  there  is  greater  strategic 
value  for  engaging  the  X  -force,  i.e.,  more  long  range  return.   Thus, 
case  A  of  Table  I  corresponds  to  where  there  is  both  more  long  range 
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and  also  short  range  return  for  engaging  X  .   Case  B  corresponds  to 
more  short  term  gain  at  the  end  of  the  battle  for  engaging  X  ,   but 
more  long  range  return  for  engaging  X  .   It  is  easily  shown  that 
case  A  results  when  Y  values  surviving  X-forces   in  direct  propor- 
tion to  their  kill  rate  against  the  Y-force,   i.e.,   p/q  =  b  /b  . 
A  switch  in  tactics  (target  priority)  is  seen  to  occur  for  this  model 
only  when  value  is  not  assigned  to  survivors  of  a  target-type  in 
proportion  to  their  destructive  capability  (kill  rate) . 

The  maximum  principle  may  be  interpreted  as  saying  that  a 
target  type  from  several  alternatives  is  engaged  when  such  an  engage- 
ment yields  the  greatest  favorable  effect  on  battle  outcome  per  unit  time, 
turns  out,  tho   1,  that  the  evolution  of  target  enga0ement  return  is 
dependent  upon  the  scenario  chosen  for  the  study  of  the  problem.   This 
is  clearly  seen  when  we  examine  the  "fight  to  the  finish."  This  is  a 
special  case  of  a  terminal  control  battle  (the  combat  ends  only  when 
the  course  of  battle  has  been  steered  to  a  prescribed  end  state)  and 
is  chosen  for  mathematical  convenience. 
2.2.   Terminal  Control  Battle. 

We  consider  the  similar  problem  of 

maximize  ry(T)  -  px  (T)  -  qx  (T)   with  T  unspecified 
0(t) 

subject  to:   -r— -  -   -  0  ay 

dx2 

o^  =  "  Vl  "  b2X2 
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x  ,x  ,y  ^  0,   0  s£  0  «  1,   and  with  terminal  states 
defined  by   (1)   x  (T)  =  x  (T)  =  0  and   (2)   y(T)  =  0. 
The  terminal  surface  of  this  problem  is  seen  to  consist  of  five  parts: 


C±:  x1(T)  =  0,  x2(T)  >  0,   x3(T)  =  0 

C2:  xx(T)  =  0  before   x  (T)  =  0,  x  (T)  >  0 

C3:  x1(T)  =  0  after   x  (T)  =  0,  x  (T)  >  0 

C4:  xx(T)  >  0,  x2(T)  =  0,   x3(T)  =  0 

C5:  x±(T)    >  0,  x2(T)  >  0,   x^T)  =  0 

The  above  problem  was  first  studied  by  Isbell  and  Marlow  [16], 
but  a  complete  solution  was  first  obtained  by  us  in  a  previous  paper 
[26] .   We  have  also  extended  the  solution  principles  to  terminal 
control  differential  games,  for  which  we  have  outlined  a  general 
solution  procedure  and  have  used  it  to  solve  the  supporting  weapon 
system  game  of  H.  K.  Weiss  [28].   The  solution  to  this  problem  appears 
complex  (see  Table  I  of  [26])  but  may  be  described  in  a  particularly 
simple  fashion  (for  the  nonrestrictive  assumption  that   a  b   >  a_b  ) . 

In  contrast  to  the  battle  of  prescribed  duration,  the  optimal 
target  engagement  may  depend  on  initial  force  levels.   When  Y  wins, 
he  engages  X   until  depletion  before  X  .   When  Y   loses,  he  may 
switch  from  firing  at  X   entirely  to  firing  at  X   entirely  before 
the  X   force  has  been  annihilated.   This  happens  when  survivors  of 
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force-type  X„   are  assigned  utility  in  excess  of  their  kill  rate  as 
compared  with  force-type  X  ,   and  certain  relationships  hold  between 
initial  force  strengths.   This  dependence  of  the  optimal  allocation 
on  initial  strengths  has  been  caused  by  the  fact  that  values  of  dual 
variables  at   t  =  T   are  dependent  upon  values  of  the  state  variables. 
This  happens  in  terminal  control  attrition  problems  where  a  value  of 
a  state  variable  is  specified  at  the  terminal  surface  (and  hence  the 
value  of  the  corresponding  dual  variable  is  unspecified  but  may  be 
determined  from  the  transversality  condition  H(t  =  T,x,p,0)  =  0, 
where  H(t,x,p,0)   denotes  the  Hamiltonian). 
2.3.   Prescribed  Duration  Battle  with  Several  Target  Types. 

We  have  considered  th    ^st  two  problems  in  ord*.   Lo  contrast 
the  effect  of  the  battle  termination  conditions  upon  the  structure  of 
the  optimal  allocation  policies.   Another  factor  that  we  can  examine 
is  the  number  of  target  types.   For  the  prescribed  duration  battle, 
certain  facets  which  tended  to  be  obscured  in  the  scenario  with  two 
target  types  are  brought  into  sharp  focus.   In  a  companion  paper  [30] 
we  present  the  supporting  analysis  details,  and  in  this  section  we 
summarize  solution  properties  in  order  that  they  may  be  contrasted 
with  those  of  other  scenarios  in  this  first  sequence  of  target  selec- 
tion problems. 

We  consider  the  following  prescribed  duration  problem: 


B-13 


n 

maximize  vy(T)  -   £   w.x.(T)   with  T   specified 

0. (t)  i=l  X   X 

dx. 

subject  to:   - —  =  -  rt.a.y   for   i  =  l,...,n 
dt      *i  1 


£  =  - !  "a 

i=i 


x-.y  iO,   0  £  0,   and   I     0.        1, 

i=l 


n 


where  all  symbols  are  used  in  the  same  sense  as  previously.   The 
solution  to  this  problem  turns  out  to  be  a  generalization  of  that  in 
section  2.1.   However,  certain  aspects  receive  greater  emphasis  to 
provide  us  with  a  deeper  understanding  of  the  phenomena  under  study. 
In  particular,  we  considered  two  subcases,  denoted  as  case  A  and  case 
B,  in  the  solution  to  the  previous  problem  of  section  2.1.   (see  Table 
I).   When  there  are  several  target  types,  the  generalization  of  the 
subcases  which  we  must  distinguish  is  as  follows: 

Case  A,   enemy  survivors  valued  in  direct  proportion  to  their 

kill  rate  against  Y-force, 
Case  B,   enemy  survivors  not  valued  in  direct  proportion  to 

their  kill  rate  against  Y-force. 

In  the  first  instance,  case  A,  it  may  be  shown  that  target 
priorities  keep  their  same  relative  ranking  over  time.   Assuming  for 
the  moment  that  no  force  type  is  totally  annihilated  during  the  course 
of  battle,  the  optimal  allocation  policy  is 
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0*(t)  =  <S    for  0  <   t  £  T, 

where   6..   is  the  Kronecker  delta  and  is  equal  to   1   if   i  =  i   and 

zero  otherwise,  and   i   is  the  index  such  that   a.b.  =  max  (a..  bn  ,  .  .  .  ,a  b  ) 

J  J         linn 

If  the  highest  priority  target  type  is  exterminated  during  such  a  battle, 
then  fire  is  merely  shifted  to  the  next  highest  priority  target.   Hence, 
when  one  values  enemy  survivors  in  proportion  to  their  kill  rate  against 
you,  i.e.,   w.  =  kb .   for   i  =  l,...,n,   the  optimal  tactic  is  to  con- 
centrate all  fire  on  a  single  target  type  until  it  is  entirely  destroyed. 
The  sole  criterion  for  target  selection  in  this  instance  is  the  quantity 
a.b.,   which  may  be  interpreted  to  be  the  rate  of  destruction  of  enemy 
attrition  capability  for  his   i    force  type  (see  section  2.1.). 

In  case  B,  it  may  be  shown  that  there  will  be  at  least  one 
switch  of  target  priorities  if  the  battle  lasts  long  enough.   Again 
assuming  that  no  force  type  is  totally  annihilated  during  the  course 
of  battle,  it  may  be  shown  that  for  t      ^  T   the  optimal  allocation 
policy  is  given  by 


0*(t)  =6.    for   0  £  t  £  T, 
l       xn 

where  j   is  the  index  such  that   a.w.  =  a.[-p.(t=T)]  =  max  (anw  , ...,a  w  ) 

3    3  3         3  11      nn 

and  we  have  arranged  our  indexing  so  that  this  is  the  last  index,  i.e., 

j  =  n,   p.(t)   is  the  dual  variable  corresponding  to  the  state  variable 

x.,   and   t.  ,   the  "backwards"  time   t  =  T  -  t   (measured  from  the  end 
J         1 

of  battle)  of  the  first  switch  in  target  selection,  will  be  given  below. 
The  battle  must  last  longer  than  t    for  a  switch  in  target  priorities 
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to  occur.   Assuming  that  it  does,  we  now  state  what  conditions  are 
necessary  for  a  change  in  target  selection  and  the  "backwards"  time 
at  which  the  change  occurs,   T  . 

In  developing  the  solution  to  this  problem  we  work  backwards 
from  the  end  of  battle,   t  =  T.   Let   k  be  the  index  of  the  target 
type  to  which  fire  is  first  shifted  in  "backwards"  time,   x.   Then  it 

may  be  shown  that  necessary  conditions  for  fire  to  be  switched  to  the 

vth  u  bk   Wk    • 

k    target  type  are  that   a.  b,  >  a  b   and  - —  >  — ,   i.e.,  we  shift 

k.  K    n  n       b    w 

n    n 

fire  to  a  target  type  which  causes  attrition  in  a  greater  proportion 
than  the  ratio  of  values  placed  upon  survivor  from  the  target  type 
which  yields  the  greatest  direct  return  at  the  end  of  battle.   The 
target  type  to  which  fire  is  shifted  has  index  k  determined  by 


R  =  min  (R  ,...,R   ), 

k   R.>0    l      n_1 

a.b .>a  b 
11  n  n 

where 

a.  (b  .w  -b  w.  ) 
_     1   1  n  n  1     r  ,       , 

R.  =  — r - ,   for   l  =  l,...,n-l. 

l   a.b .-a  b 
11  n  n 

The 'backwards"  time  of  switch  of  fire  to  the  k    target,   t1 ,   is 

determined  from  the  transcendental  equation 


/b  a.  (b.  w  -b  w.  ) 

u  ) — v—  ./    n      •   u  1 — TT~  k     k  n     n  k 

w     cosh/a  b      t,    +  v/  —  smh/a  b      t,    ■  r r~ — 

n       nnl      a        nnl     a.b-ab 


n  k  k  n  n 

This  is  seen  to  be  a  generalization  from  the  problem  with  two  X-force 
target  types. 
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Barring  the  extinction  of  a  force  type,  the  optimal  allocation 
policy  for   x,  <  T   but   t «  ^  T   is  given  by 

0*(t)  =  6.,   for   0  <  t  <   T  -  t 

0*(t)  =  6.    for  T  -  T   i  t  £  T, 

where  x    is  determined  by  (1)  and  a  similar  expression  exists  for 

x^,   the  "backwards"  time  of  the  second  change.   A  similar  solution 

with  two  changes  in  the  optimal  allocation  policy  exists  when  \     £  T 

but  x^>  >  T.   The  number  of  switches  in  target  engagement  depends  on 

the  length  of  battle,   T,   and  as  the  battle  progresses  forward  in 

time  fire  is  always  shifted  to  a  target  type  for  which  both  a.b. 

b  . 

and  —  are  smaller.   If  the  battle  lasts  long  enough,  then  during 
w . 

l 

the  initial  stages  all  fire  is  concentrated  on  the  X-force  type 

b. 

which  both  the  quantities   a.b.   and  —  are  larger  than  any  other. 

1  1        w. 

l 

Although  the  details  differ,  the  optimal  allocation  policies 
are  seen  to  have  the  same  structure  as  that  for  just  two  target  types 
(see  section  2.1.)  : 

(1)  concentration  of  all  fire  on  one  target  type, 

(2)  independence  of  allocation  from  force  levels. 

The  addition  of  more  target  types  has  not  changed  the  nature  of  the 
problem.   This  problem's  explicit  solution  is  a  generalization  of  that 
with  two  X-force  target  types. 

It  is  of  interest  to  ask  whether  the  optimal  tactic  will  always 
be  to  concentrate  fire  on  only  one  target  type  (bang-bang  optimal 
control) .   The  answer  to  this  question  turns  out  to  be  "no"  as 
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consideration  of  a  "linear-law"  attrition  process  for  the  X-force 
target  types  will  show  in  section  2.5.   An  extensive  heuristic  dis- 
cussion of  the  principles  involved  has  been  given  in  section  2.1. 
2.4.   Some  Special  Cases  of  Time  Dependent  Attrition  Rates. 

In  the  previous  idealizations  of  combat  that  we  have  considered 
above,  we  have  assumed  that  all  the  Lanchester  attrition-rate  coeffi- 
cients were  constants.   In  reality,  this  coefficient  depends  on 
numerous  factors  some  of  which  are  as  follows:   hit  probabilities, 
weapon  system  projectile-target  lethality  characteristics,  rates  of 
fire,  rate  of  target  acquisition.   These  factors  themselves  may  be 
range  dependent  or  change  over  time.   S.  Bonder  [  6  ],  [  7  ]  has  devel- 
oped explicit  formulas  for  relating  the  Lanchester  attrition-rate 
coefficient  to  weapon  system  performance  characteristics  such  as  those 
mentioned  above. 

Thus,  it  seems  appropriate  to  examine  idealized  combat  situations 
in  which  the  attrition  rates  are  time  dependent.   Solutions  have  been 
obtained  to  variable-coefficient  Lanchester-type  equations  for  a  square- 
law  attrition  process  between  two  homogeneous  forces  under  very  general 
circumstances  [29].   However,  such  solutions  are,  in  most  instances, 
so  complicated  that  they  cannot  be  applied  to  corresponding  optimal 
control  problems  to  yield  easily  interpretable  analytic  results  which 
provide  insight  into  the  structure  of  the  optimal  allocation  policies. 
There  is  a  class  of  variable  coefficient  Lanchester-type  equations 
(combat  between  two  homogeneous  forces  when  the  attrition  rates  are 
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variable  provided  that  their  quotient  is  a  constant)  which  possess  a 
solution  no  more  complicated  than  the  solution  to  the  constant 
coefficient  case  [27].   Hence,  when  one  considers  optimal  control 
problems  for  Lanchester  combat  between  forces  with  time  dependent 
attrition  rates,  there  are  some  special  instances  of  practical 
interest  (reflecting  the  physical  situation  in  which  two  weapon 
systems  cause  attrition  in  a  proportional  fashion  at  all  times)  that 
are  not  much  more  mathematically  complicated  than  the  idealized 
situations  we  have  considered  in  sections  2.1  and  2.3  above.   Again, 
we  shall  merely  summarize  results,  leaving  the  details  for  a  companion 
paper  [30] . 

We  consider  the  following  prescribed    ation  battle: 

maximize  ry(T)  -  px  (T)  -  qx  (T)   with  T   specified 
0(t) 

dxl 
subject  to:  ^—  =  -  0ax(t)y 

dx2 

dT  =  -d-0)a2(t)y 

£  =-b1(t)x1-b2(t)x2 

x  ,x  ,y  ^  0  and  0  £  0   £  1. 

We  assume  that  both  X-force  weapon  systems  are  such  that 
bx(t)  =  1^  h(t)   and  b2(t)  =  1^  h(t). 

In  the  special  case  when  the  Y-force  values  surviving 
X-force  types  in  direct  proportion  to  their  kill  rate  against  the 
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Y-force,   i.e.,   p/q  =  1^/1^   =  b^O/b^t)  =  b±  (t=T)  /b2  (t=T)  ,   the 
optimal  control  law  takes  a  particularly  simple  form 


0*(t)  = 


1  for  a1(t)b1(t)  >  a2(t)b2(t) 


0   for  a1(t)b1(t)  <  a2(t)b2(t) 


In  this  instance  target  selection  depends  only  on  the  product  of 
attrition  rates  which  may  be  interpreted  as  the  rate  of  destruction 
of  enemy  kill  rate.   All  fire  is  concentrated  on  one  of  the  target 
types  depending  on  which  target  type  has  the  larger  attrition-rate 
product.   Target  priority  is  subject  to  change  over  time  as  the  rank- 
ing of  the  target  types  on  this  decision  criterion  changes.   It  is 
conceivable  that  the  optimal  tactic  may  be  to  shift  fire  from  one 
target  type  to  the  other  several  times  over  the  course  of  battle  with 
the  duration  of  battle  not  having  any  effect.   Observe  that  no 
assumptions  at  all  have  been  made  on  the  Y-force  attrition  rates 
against  X,   and  X9  ,   i.e.,   a  (t)   and  a  (t) . 

At  this  point  we  further  assume  that  a, (t)  =  k  h(t)   and 

1       al 
a„(t)  =  k  h(t).   This  means  that  not  only  is  the  ratio  of  the  X-force 

I  a2 

weapon  system  attrition  rates  against  the  Y-force  constant  but  also 

the  ratio  of  the  Y-force  effectiveness  against  each  of  the  two 

X-force   types.   Furthermore,  all  four  attrition  rates  have  the  same 

time  dependence  except  for  constant  factors.   The  solution  is  shown 

in  Table  II.   In  this  special  case  (under  the  assumptions  noted  above), 

we  see  that  the  structure  of  the  optimal  allocation  policies  when  the 

attrition  rates  are  variable  is  essentially  identical  to  that  when 


II 


B-20 


CM 


*f 


co 

CO 

^ 

CM 
Cfl 

CU 

T3 

^ 

■U 

c 

I 

n) 

CO 

rH 

H 

1 

Pi 

H 

M 

cr 

rH 

rH 

o 

CM 

1 

4-1 

^ 

XI 

•H 

H 

H 

^w' 

rH 

4-1 

V) 

rH 

CO 

•H 

i— 1 

V! 

VI 

CO 

M 

5-i 

£> 

rH 

M 

^w 

4-J 

X 

4-1 

4-1 

H 

J-J 

H 

II 

<U 

II 

VI 

VI 

VJ 

1 

C 

o 

r*i 

CD 

^— \ 

o 

o 

H 

•H 

H 

rH 

4-1 

4-1 

4-1 

T3 

X 

v— ' 

CO 

r^ 

CO 

CN 

VI 

5-i 

5-i 

5-1 

3 

H 

•H 

Xi 

O 

O 

O 

cr 

>-^ 

5-4 

o 

m 

4-1 

4-1 

cu 

Xi 

CO 

y~^. 

H 

H 

e 

cu 

rH 

> 

4-1 

H 

O 

o 

A 

o 

V 

rH 

o 

rH 

CO 

4-1 

H O 

X 

r-l 

x> 

4-1 

£l 

4.) 

IW 

H 

ii 

rH 

II 

II 

c 

CM 

o 

•H 

c 

r- 

H 

CU 

5-i 

IS 

„ 

CM 

O 

<»"s 

**"N 

/— N 

T3 

^ 

P-. 

CM 

^ 

O 

rH 

4J 

4-J 

4-1 

a 

CM 

a 

CO 

5-1 

^> 

5-i 

^^ 

V*** 

CU 

cO 

C 

o 

M 

CM 

rH 

II 

O 

* 

O 

-X 

■K 

o 

M 

o 

•H 

^-^ 

cO 

CO 

4-1 

■e. 

4-1 

•©■ 

•G- 

CO 

s> 

•H 

4-1 

rH 

M 

e 

•^^ 

c 

*-^ 

4-1 

CO 

CO 

•H 

4-1 

cO 

x 

O 

5-i 

^J 

4-) 

^-^ 

•-> 

/""s 

5-1 

CO 

CU 

3 

A 

a 

* 

cO 

,Q 

4-1 

o 

rH 

Q 

II 

rH 

o 

■G- 

'-— ' 

>— ' 

o 

C/D 

13 

4-1 

rH 

CU 
4J 

cr 

4-J 

Xi 

v— ' 

CO 

+ 

CU 

•H 

CM 

4^ 

E 

00 

5-i 

cfl 

o 

r-*-» 

5-1 

a 

- — 

5-i 

H 

cfl 

co 

/»-s 

•• 

4-i 

H3 

H 

a) 

S-i 

4-1 

e 
o 

T3 

o 

04 

iH 

•H 

0) 

^^ 

•u 

4-1 

CO 

4-J 

■H 

x: 

o 

o 

§ 

cr 

CM 

cr 

CM 

e 

5-i 

rH          o 

r*                 ~~\ 

•H 

cu 

c 

CO 

CO 

cO 

CU 

4-1 

rH 

O 

CO 

M 

M 

4-J 

CM 

3 
rH 

4J 
4-1 

•H 
4J 

CO 

Ai 

V 

CU 

f 

f 

O 

CO 

CL 

cu 

CM 

C/3 

PQ 

1 

> 

•H 

ex 

rH 

ex 

rH 

CO 
•H 

j 

CO 

CO 

4-1 

CO 

CO 

S. 

• 

CO 

a 

M 

^S 

----' 

H 

CO 

■H 

rH 

43 

rH 

rH 

4-J 

.. 

,# 

H 

5 

w 

CO 

CO 

<d 

PQ 

CO 

rJ 

■H 

CU 

•  • 

PQ 

U 

M 

CU 

CU 

cu 

CM 

CM 

H 

CU 

a 

CO 

o 

25 

CO 

cO 
O 

CO 

co 
u 

4-1 

o 
53 

-^ 

CO 

B-21 


they  are  constant.   Only  the  time  scale  has  been  transformed  (in  [27] 
we  first  made  this  type  of  observation) . 

The  extensions  to  more  than  two  target  types  in  each  of  the 
two  cases  considered  above  possess  essentially  the  same  solution 
details  as  those  noted  here.   These  extensions  are  similar  to  those 
we  considered  in  section  2.3. 
2.5.   Selection  of  Targets  Undergoing  "Linear-Law"  Attrition. 

So  far  the  state  equations  have  described  combat  according 
to  the  Lanchester  square  law  in  which  attrition  of  a  target  type  is 
proportional  to  the  number  of  each  force  type  firing  at  it.   Weiss 
[  31]  has  given  a  thorough  discussion  of  the  conditions  which  lead  to 
this.   These  conditions  include  that  "each  unit  is  informed  about  the 
location  of  the  remaining  opposing  units  so  that  when  a  target  is 
destroyed,  fire  may  be  immediately  shifted  to  a  new  target."   It  is 
noted  that  the  control  theory  models  which  we  have  considered  so  far 
have  implicitly  assumed  perfect  information. 

Another  model  for  attrition  is  the  Lanchester  linear  law  in 
which  the  average  decrease  of  a  target  type  is  proportional  to  the 
product  of  the  average  number  of  targets  remaining  and  the  number  of 
each  force  type  firing  at  it.   Such  a  dependence  can  arise  under  two 
general  circumstances:   (i)  fire  is  uniformly  distributed  over  a 
constant  target  area  ("area  fire")  or   (2)  the  mean  time  of  target 
acquisition  is  much  larger  than  target  destruction  time  and  is  inversely 
proportional  to  target  density.   The  first  circumstance  corresponds 
to  the  simplest  case  of  partial  information.   Again  quoting  Weiss  [31], 
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we  assume  that  units  are  informed  about  the  general  areas  in  which 

opposing  units  are  located,  but  are  not  informed  about  the  consequences 

of  their  own  fire.   Thus,  we  see  that  we  may  account  for  some  changes 

in  the  information  set  by  modifying  the  description  of  combat.   Brackney 

[  8  ]  has  shown  that  "aimed  f  ire1'  may  lead  to  linear-law  attrition  when  target 

acquisition  times  are  considered,  and  are  as  postulated  above. 

Thus,  we  consider  the  following  problem  in  which  the  X-forces' 
attrition  obeys  a  linear-law  process  and  the  Y-forces'  attrition  obeys  a 
square-law  process: 

minimize  ry(T)  -  px1 (T)  -  qx  (T)   with  T  specified 

0(t) 

dxi 

subject  to:   -7—  =  -  0a- xy 

dx2 

—  =  -(l-0)a2x2y 

&     =  -  blX;L  -  b2x2 

x  ,x  ,y  ^  0  and  0  £  0   £  1. 

In  a  future  paper  we  shall  present  the  analysis  details  upon 
which  our  present  summary  is  based  (see  also  pp  91-105  of  [25]).   Since 
the  state  and  adjoint  equations  do  not  readily  yield  an  analytic 
solution,  we  have  not  been  able  to  obtain  explicit  expressions  for 
certain  model  parameters.   However,  we  can  still  discuss  all  the  quali- 
tative characteristics  of  the  structure  of  the  optimal  allocation 
policies. 

* 
See  Appendix  D. 
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There  is  a  fundamental  difference  between  the  solution  to 
this  problem  and  those  considered  previously:   the  optimal  allocation, 
0* ,   may  be  other  than  0  or   1.   In  contrast  to  the  previous 
problems,  the  optimal  allocation  policy  does  not  have  to  be  an  extreme 
point  of  the  control  variable  space  at  all  times.   We  may  have  a 
singular  solution  [18]  for  which  the  necessary  condition  of  maximizing 
the  Hamiltonian  (with  respect  to  the  control  variable)  does  not  pro- 
vide us  with  a  well-defined  expression  for  the  extremal  control.   We 
shall  call  the  part  of  an  optimal  trajectory  on  which  the  maximum 
principle  cannot  be  used  to  determine  the  control  a  singular  subarc. 
Then  the  term  "singular  solution"  will  be  used  to  refer  to  any  optimal 
trajectory  which  contains  one  or  more  singular  subarcs. 

Singular  solutions  can  only  occur  when  the  Hamiltonian 

(denoted  as  H)  is  a  linear  function  of  the  control  variables.   When 

3H 
this  is  so,  then  if  —  =  0   for  a  finite  interval  of  time  (or, 

00 

another  way  to  say  this,  the  coefficient  of   0  vanishes  identically 

for  a  finite  interval  of  time) ,  the  maximum  principle  does  not 

3H 
determine  the  control.   Observe  that  when  t—  =  0  and  H  is  a  linear 

30 

function  of   0,   all  feasible  values  of   0  are  optimal.   All  problems, 
however,  for  which  the  Hamiltonian  is  a  linear  function  of  the  control 
variables  do  not  have  singular  subarcs  in  their  solution.   For  example, 
it  may  be  shown  that  such  a  singular  control  is  impossible  for  the 

problems  considered  above  in  sections  2.1  through  2.4,  since  it  is 

■jH 


impossible   for     — 
at 


M) 


3H 
0   for  a  finite  interval  of  time  when  —  =  0, 

o0 
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For  the  problem  at  hand,  it  may  be  shown  that  a  necessary  condition 
for  a  singular  subarc  to  yield  the  maximum  return  [20]  is  satisfied. 

The  optimal  battle  trajectories  are  constructed  by  working 
backwards  from  all  possible  end  points  of  this  idealized  battle.   Con- 
sideration is  given  to  both  the  optimal  control  at  the  end  of  battle 
and  also  how  the  variables  upon  which  it  depends  vary  over  time. 
Based  upon  such  considerations,  it  may  be  shown  that  there  are  three 
cases  to  be  considered: 

b 

Case  (a)   —  =  - —  , 
q   b2 

b 

Case  (b)   ^  >  ~   , 

q  b2 


b 

Case  (c)   £-  <  - —  . 

q   b2 


We  consider  Case  (a)  first.   The  solution  for  this  case  is 
shown  in  Figure  1.   Even  though  explicit  expressions  have  not  been 
obtained  for  certain  model  parameters,  the  dependence  of  the  optimal 
control  upon  these  quantities  can  still  be  qualitatively  discussed. 
It  may  be  shown  that  the  optimal  control  depends  on  the  state  variables 
x   and  x    (and  also  the  attrition  coefficients)  in  each  "decision 
region."  Above  the  line  a  b  x  =  a  b  x  ,   denoted  as  L,   the  optimal 

control  0*   =  0   is  used  until  this  line  is  encountered.   When  L   is 

a2 
reached,  the  singular  control  0*   =   — ,   which  keeps  the  trajec- 

al  +  a2 

tory  on  L,   is  used  until  the  end  of  the  battle  at   t  =  T.   That 


B-25 


CVJ 

o 

CVJ 
CVJX 
O       CM 

\  * 

II 

X 

XI 

-J 

< 
it  k 

\  m 

Q 

o 

> 

: 

II 

* 

ii 

V° 

y 

t      » 
* 

t 
i 
i 

1 

-1  w 

ii 

>-• 
ii 

Li. 

Q.  |o" 

O 

V) 

o 
O 

~< 

i 

c 
o 


o 

C 


o 
o 
o 


O 

E 

Q. 

o 


CM 
X 


B-26 


portion  of  an  optimal  trajectory  which  lies  on  L   (for  a  finite 
interval  of  time)  is  a  singular  subarc.   The  above  type  of  solution 
holds  for  arbitrary  initial  values  of   x   and   x  :   x  (t=0)  ■  x 
and  x  (t=0)  =  x  .   The  time  history  of  the  optimal  control  is  traced 

for  two  particular  initial  force  ratios  denoted  as  points  A  and  D 

Xl  a2b2 

in  Figure  1.   At  point  D,  —tt  >  — r—  and  <rt*  =  1   is  used  until  the 

0  a  b 

X2  11 

line  L   is  encountered  at  point   E. 

The  solution  for  Case  (b)  is  shown  diagrammatically  in  Figure 
2.   It  is  similar  to  the  preceding  case  except  that  another  line,   L' 
with  equation  a  px  =  a_qx9,   plays  a  central  role  besides  the 
singular  "surface"  denoted  as  L.   The  final  segment  of  that  part  of 
any  optimal  trajectory  which  lies  below  L'   consists  of  the  use  of 
the  optimal  control  0*  =  1   for  the  final  stages  of  battle.   Such  is 
the  case  for  the  optimal  trajectories  denoted  as  (2),  (3),  and  (4). 
Any  optimal  trajectory  which  lies  entirely  above  L',   such  as  (1), 
has  a  corresponding  optimal  control  of   0*(t)  =  0   for   0  £  t  £  T, 
whereas  a  similar  remark  holds  for  any  one  that  lies  entirely  below 
L,   such  as  (5).   Case  (c)  is  symmetric  to  Case  (b) . 

As  has  been  noted  previously  (see  sections  2.1  and  2.3),  the 
structure  of  the  optimal  allocation  policies  in  these  tactical  alloca- 
tion problems  is  dependent  upon  how  the  Y   force  values  the  surviving 

X-force   types  relative  to  their  kill  rate  against  the  Y   force. 

b1 

Case  (a)  :   ■*-  =  ; —  above  is  when  Y  assigns  utility  to  surviving 
q   b2 

X-force   types  in  exact  proportion  to  their  destructive  capability 
against  Y.   In  this  case,  the  optimal  target  selection  tactic  depends 
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only  upon  the  state  of  the  system  as  is  seen  with  reference  to  Figure 
1.   The  optimal  tactic  is  to  use  0*(t)  =  0  above  the  line  L  with 
equation  a  b  x  =  a  b  x  .   The  line  L  also  represents  an  "equili- 
brium" trajectory  which  the  system  follows  whenever  this  line  is 
reached. 

Case  (b)  :  ■*-  >  —  is  when  Y  assigns  a  greater  value  to 
q   t>2 

surviving  X  '  s   than  in  proportion  to  their  kill  rate  against  Y 
relative  to  that  of  X  .   Again,  the  optimal  tactic  depends  upon  the 
state  of  the  system,  only  this  dependence  itself  depends  upon  the 
"time  phase"  of  the  fixed  length  battle.   There  is  a  x    such  that 
for  0  «£  t  ^  t  ,  where  x   is  "backwards  time"  measured  from  the  end 
of  v-"tle  x  =  T  -  t,   the  optimal  tactic  depends  upon  the  s  ite  of 
the  system  relative  to  the  line  L'   with  equation  a  px  =  a.qx? 
(see  Figure  2).   Below  L'   the  optimal  tactic  is  to  use  0*(t)  =  1. 
For   t  ^  t  «£  T,   the  optimal  tactic  is  to  use  0*(x)  =  0  above  the 
line  L  with  equation  a  b  x  =  a_b  x  .   We  recall  that  the  part  of 
an  optimal  battle  trajectory  that  coincides  with  L   is  a  singular 
subarc.   It  may  be  shown  that  the  "crossover  time,"   x  ,   depends 
upon  the  particular  battle  trajectory  under  consideration. 

Based  on  our  examination  of  the  above  problem,  we  see  that  the 
structure  of  the  optimal  allocation  policies  for  targets  which  undergo 
a  linear-law  attrition  process  has  the  following  characteristics: 

(1)  possible  division  of  fire  between  target  types, 

(2)  dependence  of  allocation  upon  force  levels. 
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These  characteristics  should  be  contrasted  to  those  previously  observed 
with  respect  to  a  square-law  attrition  process  (see  sections  2.1  and 
2.3).   When  there  is  a  linear-law  attrition  process  for  target  types, 
we  may  have  other  than  0  or   1  as  the  optimal  allocation  policy. 
Also,  the  allocation  depends  upon  the  force  levels  of  target  types. 
We  have  attempted  to  explain  this  structure  of  the  optimal  allocation 
policies  in  terms  of  the  nature  of  the  attrition  process  in  section 
2.1. 

3.   Regulation  of  Firing  Rate. 

In  this  section  we  will  examine  a  sequence  of  models  of 
increasing  complexity  for  which  the  effect  of  ammunition  limitations 
on  firing  rate(fire  discipline)  will  be  explored.   In  each  case,  we 
consider  two  homogeneous  forces  engaged  in  combat  described  by  a  square 
law.   Our  results  for  these  problems  are  of  a  more  preliminary  nature 
than  those  reported  above  for  target  selection. 

3.1.   Battle  of  Prescribed  Duration  with  Constant  Attrition  Rates. 
We  consider  the  problem 


maximize  px(T)  -  qy(T)   with  T   specified 
0(t) 

dx 
subject  to:  —  =  -  a  y 
J         dt      1 


dy 

dt  =  "  *VV 


dz 
dt 


=  $v 


,y  £  0,   0*0*1,   z(t=0)  =  0  and  z(t=T)  £  A  <  vT  =  v 


dt, 
^0 
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where  v  is  the  maximum  firing  rate  of  each  X  unit.  It  should  be 
noted  that  the  nature  of  the  attrition  coefficients  a  and  a  is 
different,  since  a   has  incorporated  in  it  a  firing  rate. 

We  have  assumed  that  each  X   combatant  has  a  limited  supply 
of  ammunition  (his  "basic  load") ,  denoted  by  A.   There  is  no  redis- 
tribution of  ammunition  during  the  course  of  battle,  i.e.   when  a 
casualty  occurs,  his  ammunition  is  lost  to  the  other  combatants.   We 
further  assume  that  the  ammunition  supply  is  such  that  a  combatant 
could  not  fire  at  his  maximum  firing  rate  for  the  prescribed  duration 
of  the  battle,  for  when  A  ^  vT   it  is  easily  seen  that  the  optimal 
strategy  is  to  fire  at  the  maximum  possible  rate,   0*(t)  =  1  for 
£  t  *  T. 

The  optimal  regulation  of  firing  rate  turns  out  to  be 

A 
0*(t)  =  1   for   0  £  t  £  T   where  T  =  -  , 

0*(t)  -  0   for  T  £  t  £  T. 

This  was  determined  as  follows.   The  Hamiltonian  is  given  by 

H(t,xi,pi,  )  =  0V(p3"P2a2x)  -  P1a1YJ 

where  p.(t)   is  the  dual  variable  corresponding  to  the   i    state 
variable  (we  denote  x  as   x  ,   y  as   x  ,   z  as   x  ).   Maximization 
of  the  Hamiltonian  leads  to  the  following  control  law 

(  0   for   (-p.(t))a0x  <  A* 


0*(t)  = 


1   for   (-p2(t))a2x  >  X*  >  0, 
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where  p~(t)  =  -  X*.   The  adjoint  differential  equations  are  given 


by 


^1     3H 
dt 


3^  -  0va2P2   with   P1(t=T)  =  p 


dT=-f=alPl     With   P2(t=T)=-q 

p  (t)  =  constant      with   p  (t=T)   unspecified. 

To  develop  the  desired  solution  we  work  backwards  from  the  end  of 
battle.   Hence,  we  introduce  the  "backwards  time"  variable   t  =  T  -  t 
and  consider  a  backwards  integration  of  the  state  and  adjoint  differ- 


ential equations  from  the  fixed  end  of  the  battle,   t  =  T.   Thus, 

dp, 

-r— =  -  0  va  p  ,   etc.   It  is  easy  to  show  that  p  (x)  ,   x(t),   and 


y(x)   are  non-decreasing  functions  of   t   (regardless  of  the  value  of 
0)   with  p  (x=0)  =  p,   x(t=0)  =  x  ,   and  y(x=0)  =  y  .   Similarly, 
p_(x)   is  a  strictly  decreasing  function  of   x.   Hence,   Q(t)  = 
(-p„(x))a  x   is  a  strictly  increasing  function  of   x  with  an  initial 
value  of  Q(t=0)  =  qa  x  .   Thus,   p~  must  be  negative,  and  0*(x) 
never  switches  back  to   0  once  it  becomes   1. 

The  optimal  tactic  for  this  model  is  disturbing,  since  it  is 
not  intuitively  appealing  to  fire  at  one's  maximum  firing  rate  until 
one  runs  out  of  ammunition  and  to  spend  the  final  stages  of  battle 
without  ammunition.   However,  after  a  little  reflection  we  realize 
that  it  is  this  model's  assumption  of  constant  attrition  rates  which 
has  negated  the  holding  of  one's  fire  as  an  optimal  tactic.   Hence, 
we  are  led  to  consider  other  models  for  further  insight. 
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3.2.   Battle  of  Prescribed  Duration  with  Time  Varying  Kill  Rates. 
We  consider  the  problem 


maximize  px(T)  -  qy(T)   with  T   specified 
0(t) 

dx 
subject  to:  —  =  -  a1(t)y 


-£  "  "  0va2(t)x 


dz 

dT=  ^ 

x,y  ^  0,   0  s:  0  £  1,   z(t=0)  =  0  and   z(t=T)  £  A  <  vt. 

The  precise  nature  of  a  (t)  and  a  (t)  will  depend  upon  the  specific  sJ 
under  consideration.  It  seems  reasonable  to  assume  that  in  many  real 
world  situations  a1 (t)  and  a  (t)  would  be  monotonically  increasing 
functions  of  time,  e.g.,  two  forces  closing  with  one  another.  All  the 
previous  solution  steps  remain  the  same  except  for  the  effect  of  a  (t) 
and  a  (t)  increasing  with  time.  This  may  change  the  nature  of  the 
solution  markedly,  although  the  optimal  control  is  still  bang-bang. 
One  can  show,  in  fact,  that  a  singular  solution  is  impossible  for  this 
problem.  The  quantity  Q(t)  =  (-p~(x))a  (t)x(t)  is  no  longer  guaranteed 
to  be  a  strictly  increasing  function  of  t,  since  a  (t)  is  strictly 
decreasing  (but  positive) .  This  allows  the  possiblity  that  the  optimal 
tactic  may  be  to  hold  one's  fire  and  conserve  ammunition  in  the  early 
stages  of  battle  so  that   0*(t=T)  =  1  at  the  end  of  battle. 
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The  way  in  which  ammunition  is  conserved  depends  on  the 

specific  nature  of   a  (t)   and   a  (t) .   As  we  have  pointed  out  above, 

under  special  circumstances  to  solution  to  variable  coefficient 

Lanchester-type  equations  (and  also  the  adjoint  system  of  equations) 

is  no  more  complicated,  than  the  constant  coefficient  case.   If  we 

va2(i)    a2 

assume  that  3—r  =  -. ,   then  there  are  relatively  simple  expressions 

a.  v.t )        k. 

al 
for  p?(t)   and  x(t).     For  example,  when  0=1   for  0  s;  t  £  t  , 


we  have 


(x)  ~  p/~ 


P_(t)  =  -  q  cosh  6(t)  -  p/  r- —  sinh  8(t) 

a2 


where  6(t)  =  /k. k" 


al  a2 


T 


h(t)dT.   However,  even  such  relatively 
0 
simple  expressions  for  these  quantities  has  not  led  to  a  general 


statement  about  under  what  circumstances  one  holds  his  fire  even 
though  this  can  be  readily  numerically  determined  for  any  particular 
set  of  parameter  values.   A  question  of  practical  importance  to  be 
answered  by  further  investigation  along  these  lines  is,  "For  what  types 
of  attrition-rate  range  dependencies  is  it  the  optimal  tactic  to  hold 
one's  fire  until  an  enemy  gets  closer  due  to  an  ammunition  constraint." 

R.  Isaacs  has  studied  some  similar  problems  in  his  book 
Differential  Games  [15]  and  has  explored  some  aspects  much  deeper  than 
presented  here.   Isaacs  tried  to  resolve  the  problem  of  shooting  up  all 
of  one's  ammunition  before  the  end  of  the  battle  by  modifying  the  pay- 
off.  Another  approach  might  be  to  consider  a  terminal  control  problem. 
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3.3.   Fight  to  the  Finish  with  Limited  Ammunition. 

We  consider  briefly  the  constant  coefficient  problem  of  section 
3.1  only  with  T   unspecified  and  with  terminal  states  defined  by 
(1)   x(T)  =  0  and   (2)   y(T)  =  0.   In  this  case,  the  optimal  tactic 
is  dependent  upon  the  force  levels,  since  the  ammunition  constraint 
is  not  binding  for 


/  s£  x_  tanh  A/  . 

va     0      [    V  J 


Va2 


It  should  be  noted  that  the  above  condition  also  guarantees  that  X 
will  win.   Furthermore,  the  X   forces  in  order  to  win  are  required 
to  have  enough  ammunition  uo  fire  at  their  maximum  rate  during  che 
entire  duration  of  the  battle.   Hence,  we  see  that  concentration  of 
forces  reduces  the  ammunition  requirement  per  man,  since  the  length 
of  battle  is  determined  by  the  initial  numbers  of  forces  committed 
to  battle. 
3.4.   A  Two-Sided  Extension. 

It  seems  appropriate  to  contrast  qualitatively  the  structure 
of  the  optimal  regulation  of  firing  rate  for  the  above  problems  with 
a  two-sided  version,  even  though  we  do  not  attempt  to  solve 
the  latter  at  this  time.   Thus,  we  consider  the  problem 


maximize  minimize  px(T)  -  qy(T)   with  T   specified 
(pit)  \p  (t) 
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,  .         dx 
subject  to:  -—  =  -  tf;a,v,y 
dt     ^  1  1; 

dy 

dt  =  "  ^a2V2X 

du 

dt"  =  ^2 

dv  -  , 

x,y  2:  0,   0^0,   $  £  1,   u(t=0)  =  0,   u(t=T)  s  A2  <  v2T, 

v(t=0)  =  0,   v(t=T)  £  A  <  v-T. 

Unlike  the  previous  one-sided  version  of  this  problem,  it  is  now 
possible  to  have  0*(t=T)  =  1  with  limited  ammunition.   This  possibility 
has  arisen  since  the  Y  forces  may  hold  their  fire  during  the  early 
stages  of  engagement.   Questions  now  arise  as  to  the  advantage  of 
delivering  the  first  shot,  e.g.,  is  there  a  time  lag  before  fire  is 
returned?,  and  we  move  into  the  realm  of  games  of  timing  studied  at 
RAND  [19]. 

4.   Extensions  to  Differential  Games. 

Even  though  it  is  certainly  true  that  combat  is  an  environ- 
ment of  conflicting  interests  in  which  the  potential  actions  of  both 
friendly  and  enemy  forces  must  be  considered,  there  is  much  to  be 
learned  from  one-sided  dynamic  optimization  models.   We  view  these 
simplified  idealizations   presented  here  as      "building  blocks" 
for  more  sophisticated  scenarios.   We  feel,  however,  that  an  under- 
standing of  the  structure  of  optimal  tactics  for  these  initial  models 
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is  essential  before  one  continues  his  examination  of  a  sequence  of 
models  of  greater  and  greater  complexity.   Hence,  it  seems  appropriate 
to  review  the  intimate  connection  between  optimal  control  theory  and 
differential  games. 

It  has  been  stated  that  optimal  control  problems  may  be 
viewed  as  one-sided  differential  games  for  which  the  roles  of  all  but 
one  of  the  competing  players  have  been  suppressed  [3],   A  concise  dis- 
cussion of  the  inter-relationships  between  these  two  subjects  is 
contained  in  Y.  C.  Ho's  [14]  excellent  review  of  Isaacs  book  [15]  (see 
also  Chapter  9  in  [9]). 

When  one  recalls  the  equivalence  of  variational  problems  and 
partial  differential  equations  of  the  first  order  (first  pointed  cut 
by  C.  Jacobi  nearly  140  years  ago  [17]),  the  relationship  between 
optimal  control  and  differential  games  may  be  viewed  as  follows.   In 
an  optimal  control  problems  we  are  seeking  the  solution  to  the  Hamilton- 
Jacobi-Bellman  equation  for  the  optimal  value  function,  denoted  as   S 
(sometimes  referred  to  as  Hamilton's  characteristic  function  in  the 
calculus  of  variations  literature  [23]) 

3S      .    ,T/     3S   , 

- 1-  maximum  H(t,x.,- — ,<*;  =  0 

at  ,    s        i  dx . 

0(t)  1 

with  appropriate  boundary  conditions.   In  a  differential  game  we  seek 
the  solution  to  the  Bellman-Isaacs  equation  [13] 

9  S  3  S 

1-  maximum  minimum  H(t,x.,- ;0,4>)  -  0. 

ot  /  _\        ,    /  .  \  1   OX  . 

0(t)     l|>(t)  1 
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Moreover,  this  approach  may  also  be  extended  to  when  stochastic  effects 
are  present  as  has  been  done  by  S.  Dreyfus  [12]  and  others  [11],  [33]. 

It  also  seems  appropriate  to  mention  the  relationship  of 
dynamic  programming  to  these  techniques.   Consideration  of  the  equation 
satisfied  by  the  optimal  value  function  points  out  clearly  an  important 
aspect  of  dynamic  programming,  its  being  a  discrete  approximation 
technique  for  solving  variational  problems  [12].   It  is,  however,  a 
dual  approach  which  generates  an  optimal  trajectory  as  an  envelope 
of  tangents  rather  than  as  a  sequence  of  points  [1].   The  value  of 
these  continuous  models  lies  in  their  mathematical  tractability  which 
many  times  allows  us  to  develop  "closed-form"  solutions.   This  should 
be  contrasted  with  dynamic  programming  models  for  which  general  solu- 
tions are  rarely  obtained,  and  one  must  be  satisfied  with  generating 
a  numerical  solution  for  a  particular  set  of  parameter  values. 
In  this  case,  it  is  difficult  (if  not  impossible)  to  see  the  structure 
of  optimal  allocation  policies  and  its  dependence  upon  model  form 
without  a  parametric  analysis  of  model  output. 

We  have  previously  pointed  out  [26]  that  the  existing  theory  of 
differential  games  is  only  applicable  to  problems  with  pure  strategy 
solutions.   The  structure  of  two-sided  fire  distribution  problems 
in  the  Lanchester  theory  of  combat  as  formulated  by  H.  Weiss  [31]  leads 
to  such  a  result.   However,  when  defensive  capabilities  were  considered 
in  the  attrition  process  in  a  tactical  air  war  game  extensively  studied 
at  RAND,  the  resulting  model  did  not  possess  a  solution  in  pure 
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strategies  [2],  [4],  [5].   Thus,  there  are  limits  to  the  applicability 
of  such  variational  models. 

We  have,  therefore,  used  these  optimal  control  problems  to 
study  many  aspects  of  corresponding  two-sided  variational  problems: 
the  effect  of  different  boundary  conditions,  devising  solution  pro- 
cedures, study  of  singular  behavior,  differences  in  the  structure  of 
optimal  allocation  policies  for  various  model  forms.   Most  solution 
aspects  of  the  one-sided  problem  are  present  in  the  two-sided  one. 
There  are  some  significant  differences,  however.  In  solving  the  support- 
ing weapon  system  game  of  H.  K.  Weiss  [32],  we  have  encountered  solu- 
tion behavior  unique  to  terminal  control  attrition  games:   there  may 
exist  a  domain  of  controllability  for  a  given  terminal  state  but  entry 
to  this  state  may  be  "blockable"  by  the  "losing"  player  [28] .   In  other 
words,  there  is  a  path  determined  by  the  necessary  conditions  leading 
from  each  point  in  a  region  of  the  initial  state  space  to  a  terminal 
state,  but  the  "losing"  player  may  use  a  strategy  other  than  his 
extremal  strategy  for  this  path  to  actually  win.   We  made  use  of  our 
knowledge  of  a  related  optimal  control  problem  [26]  to  solve  this 
differential  game. 

5 .   Implications  of  Models. 

It  seems  appropriate  to  briefly  discuss  the  general  implica- 
tions in  the  following  areas  of  the  models  examined  in  this  appendix: 

(1)  optimal  tactical  allocation, 

(2)  intelligence, 
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(3)  command  and  control  systems, 

(4)  human  decision  making. 

Our  discussion  of  these  areas  is  not  mutually  exclusive. 

Of  interest  to  the  military  tactician  is  whether  target 
selection  rules  evolve  dynamically  during  the  course  of  battle.   Are 
target  priorities  static  or  do  they  evolve  dynamically  with  the  course 
of  battle?   With  respect  to  optimal  control  models,  this  may  be 
mathematically  stated  as  whether  there  are  transition  (switching) 
surfaces  in  the  solution.   We  have  seen  in  the  idealized  and  simplified 
models  studied  here  that  target  priorities  do  change.   This  is  related 
to  the  evolution  of  marginal  return  of  target  destruction  (value  of  dual 
variable) .   We  have  seen  that  this  evolution  depends  on  the  goals  of 
the  combatants  (utility  assigned  to  surviving  force  types  at  the  end 
of  the  battle)  and  also  the  conditions  which  terminate  the  battle.   In 
the  terminal  control  problem  studied  here,  a  shift  in  target  priorities 
is  present  only  in  a  losing  case,  whereas  in  a  fixed  duration  battle 
such  a  switch  is  independent  of  winning  or  losing  but  depends  only  on 
weapon  system  capabilities  and  the  prescribed  duration  of  battle. 

Even  though  these  models  assume  complete  and  instantaneous 
information,  it  appears  that  some  inferences  may  be  made  for  cases 
where  uncertainty  is  present.   In  the  terminal  control  case,  we  saw 
that  selection  of  tactics  depends  on  a  knowledge  of  the  enemy's  strength 
and  capabilities,  since  the  terminal  state  of  combat  must  be  determined 
before  optimal  strategies  can  be.   For  a  battle  of  prescribed  duration, 
e.g.,  fighting  a  delaying  action  in  a  regrograde  movement  to  protect 
the  withdrawal  of  troops,  tactics  depend  only  on  enemy  and  friendly 
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capabilities  and  length  of  combat,  not  the  initial  force  levels.   For 
such  cases  the  estimate  of  combat  length  is  critical,  since  changes  in 
target  priorities  are  determined  relative  to  the  end  of  the  engagement. 

Schreiber  [24]  has  proposed  an  idealized  and  simple,  but  yet 
illuminating,  way  of  quantitatively  showing  the  value  of  intelligence 
and  command  control  capabilities.   He  introduces  the  concept  of  "command 
efficiency,"  which  is  measured  by  the  fraction  of  the  enemy's  destroyed 
units  from  which  fire  has  been  redirected.   The  effect  of  poor  intelli- 
gence and  poor  capabilities  for  redirecting  fire  from  destroyed  targets 
is  to  produce  "overkill."   Schreiber' s  equations  for  combat  involved 
this  fraction  called  "command  efficiency,"  and  they  reduce  to  Lanchester- 
type  equations  for  area  fire  when  the  fraction  is   0  and  aimed  fire 
for  a  value  of   1.   We  have  seen  that  the  optimal  tactics  are  quite 
different  for  these  two  cases.   When  intelligence  and  command  control 
systems  are  very  efficient,  the  optimal  tactic  is  seen  always  to  be 
concentration  of  fire  on  a  specific  target  type.   When  capability  for 
redirection  of  fire  from  destroyed  targets  is  poor  (either  through  damage 
assessment  or  constraints  on  new  target  acquisition) ,  the  optimal  tactic 
may  be  to  allocate  fire  in  a  proportional  fashion  over  target  types  in 
a  way  that  holds  the  ratios  of  target  density  in  each  target  area  to 
be  constant.   Another  implication  is  that  supporting  weapon  systems 
(e.g.,  artillery)  concentrate  fire  on  selected  point  targets,  but  that  it 
sometimes  is  best  to  allocate  fire  proportionately  over  various  area  targets 
these  models  suggest  that  the  tactics  of  target  engagement  may  vary 
with  command  and  control  capabilities. 


B-41 


These  models  also  show  the  importance  of  intelligence  in  de- 
vising the  "best"  tactics  in  combat.   Intelligence  on  enemy  weapon 
system  capabilities  (kill  rates  including  target  acquisition  rates) 
and  potential  length  of  engagement  play  a  central  part.   We  also  have 
seen  that  for  fights  to  the  finish  and  linear  law  attrition  cases 
intelligence  on  enemy  force  levels  is  also  required.   For  artillery 
fire  support  missions  against  various  troop  concentrations,  knowledge 
of  troop  densities  is  essential  in  the  assignment  of  target  priorities. 
Particularly  dense  concentrations  where  the  initial  kill  potential  is 
high  are  seen  to  be  cases  where  the  optimal  tactic  is  to  concentrate 
fire  on  one  target  for  awhile. 

Another  argument  for  the  concentration  of  forces  is  seen  to 
emerge  from  the  study  of  these  simplified  models.   When  ammunition  is 
limited,  a  concentration  of  forces  has  the  effect  of  counter-balancing 
this  constraint.   For  example,  in  a  fire  fight  numerical  superiority 
could  mean  that  the  enemy  force  level  would  be  reduced  such  that  he 
would  disengage  in  time  before  the  friendly  ammunition  restriction 
became  critical. 

These  models  may  be  interpreted  to  show  the  value  of  human 
judgment  in  combat.   They  indicate,  as  does  common  sense  and  experience 
that  in  battle  a  commander  must  use  his  judgment  to  ascertain  to  what 
end  can  the  course  of  battle  be  steered  so  that  he  may  devise  his 
strategy  accordingly.   The  demonstrated  sensitivity  of  these  models  to 
many  factors  shows  the  importance  of  human  assessment  of  a  situation 
and  the  importance  of  good  judgment  in  assigning  utility  to  forces 
surviving  the  battle  at  hand. 
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6.   Summary. 

The  results  of  this  appendix  may  be  summarized  as  follows: 

(1)  a  sequence  of  one-sided  models  has  been  presented  which  shows 
that  the  tactics  of  target  selection  may  be  sensitive  to 
force  strengths,  target  acquisition  process,  the  type  of 
attrition  process,  and/or  the  termination  conditions  of 
combat, 

(2)  a  sequence  of  models  have  been  presented  which  shows  some 
preliminary  results  about  the  effect  of  resource  constraints 
on  firing  discipline  and  concentration  of  forces, 

(3)  tactics  for  target  selection  are  heavily  dependent  upon 
"command  efficiency," 

(4)  concentration  of  fire  always  on  one  target  type  among  many 
occurs  as  an  optimal  tactic  only  when  target  acquisition 
is  not  subject  to  diminishing  returns, 

(5)  target  priorities  don't  change  over  time  when  one  assigns 

a  worth  to  surviving  target  types  in  direct  proportion  to  their 
kill  rate  against  you. 
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Appendix  C.   Target  Selection  in  Lanchester  Combat:   Heterogeneous 
Forces  and  Time  Dependent  Attrition  Rates. 

1.   Introduction. 

In  this  appendix  we  develop  solutions  to  a  sequence  of  models 
for  optimal  target  selection  in  combat  against  heterogeneous  enemy 
forces.   The  combat  continues  over  a  period  of  time  with  a  choice 
of  tactics  available  to  one  side  and  subject  to  change  with  time. 
Optimal  target  selection  rules  are  developed  through  the  combina- 
tion of  Lanchester-type  equations  for  combat  attrition  and 
deterministic  optimal  control  theory  (Pontryagin  maximum  principle) . 
Both  constant  attrition-rate  coefficients  and  some  special  cases  of 
variable  attrition  rates  are  considered.   The  synthesis  of  optimal 
target  engagement  policies  was  facilitated  by  exploiting  special 
mathematical  structures  in  these  problems. 

Many  years  ago,  Karl  von  Clausewitz  said  that  if  theory 
caused  a  more  critical  study  of  war,  then  it  had  achieved  its 
purpose.   In  Appendix  B  we  studied  the  structure  of  the  optimal 
allocation  policies  for  some  basic  elementary  tactical  situations 
described  by  Lanchester-type  equations  of  warfare.   We  did  this  by, 
among  other  things,  contrasting  the  "best"  target  selection  policies 
for  a  sequence  of  scenarios:   prescribed-duration  battle,  terminal- 
control  battle,  two  enemy  target  types,  many  enemy  target  types,  etc. 
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In  Appendix  B  we  frequently  summarized  results  and  promised  the 
details  of  solution  development  (for  the  non-trival  cases)  to  be 
available  elsewhere. 

Thus,  the  purpose  of  this  appendix  is  to  provide  supporting 
analysis  details  for  target  selection  in  Lanchester  combat  against 
heterogeneous  enemy  force  types  and/or  combat  in  which  the  attri- 
tion rates  may  change  over  time.   All  battle  scenarios  that  we 
consider  here  are  for  a  prescribed  duration  of  time.   We  also 
present  some  more  general  results  than  those  reported  in  Appendix  B: 
target  selection  for  combat  against  several  enemy  force  types  with 
all  attrition  rates  subject  to  change  over  time. 

Again,  the  purpose  of  this  appendix  is  to  provide  analysis 
details.   The  interested  reader  will  find  an  extensive  discussion 
of  the  structure  of  optimal  allocation  policies  and  model  implications 
in  Appendix  B.   Our  presentation  in  this  present  appendix  follows 
the  historical  order  in  which  we  attacked  these  problems.   We 
could  have  merely  reported  the  solution  of  the  problem  in  section  4. 
This  would  have  been  more  concise.   We  feel,  however,  that  it  is  more 
valuable  to  show  the  reader  how  we  have  used  insight  gained  from 
simpler  problems  to  provide  guidance  for  solving  more  complex  ones 
and  generalizing  results. 

In  this  appendix  we  develop  the  solutions  to  all  problems  by 
the  mathematical  theory  of  optimal  control.   For  the  problems  at 
hand  we  need  only  make  use  of  the  Pontryagin  maximum  principle  [8] . 
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The  reader  should  note  that  there  is  a  sign  difference  between 
developments  in  this  country  (see  p.  108  of  [6]  or  pp.  12-14  of  [7]) 
and  those  in  the  Soviet  Union  [1],  [8],  although  both  approaches 
yield  exactly  the  same  results  in  applications. 

We  wish  to  impart  to  the  reader  through  our  examination  of 
these  problems  our  experience  that  some  ingenuity  is  required  to 
solve  applied  problems.   By  taking  advantage  of  the  structure  to 
some  problems,  we  have  found  that  one  can  frequently  synthesize 
the  optimal  control  to  a  problem  which  at  first  sight  appears  to 
be  too  complex  to  allow  this.   By  the  synthesis  of  the  optimal 
control,  we  mean  the  explicit  determination  of  the  time  history  of 
the  optimal  control  from  initial  to  terminal  time.   For  the  prob- 
lems at  hand,  this  is  accomplished  by  determining  a  control  law  by 
the  maximum  principle  and  then  working  backwards  from  the  end  of 
the  problem  by  a  backwards  integration  of  the  adjoint  system  of 
differential  equations  for  the  dual  variables  which  have  a  boundary 
condition  there. 

The  organization  of  this  appendix  is  as  follows.   First,  we 
consider  the  problem  of  optimal  selection  of  a  target  type  from  among 
two  alternatives  for  some  special  cases  of  variable  attrition  rates. 
Then,  we  treat  the  more  general  case  of  several  target  types  for 
some  special  instances  of  variable  attrition  rates.   Finally,  we 
make  some  observations  on  the  results  presented  in  this  appendix. 


C-4 


2.   Two  Target  Types,  Some  Special  Cases  of  Variable  Attrition 
Rates . 

We  consider  the  following  prescribed  duration  battle: 


maximize  ry(T)-px  (T)  -  qx_(T)   with  T   specified, 
0(t) 

dxl 
subject  to:  -r— -  =  -0a  (t)y, 

dx 

jf  =   -(l-0)a2(t)y, 

of  =  -*l<t)xl  "b2(t)x2' 


x  ,x  y  :>  0   and   0  tf  0  £  1,  (1) 


where 

p,  q  and   r  are  values  placed  on  surviving  force  types, 

x1 ,  x   and  y  are  average  force  strengths, 

a  (t) ,  a  (t) ,  b  (t)   and  b  (t)   are  attrition  rates  which 

are  allowed  to  change  over  time, 

and  0  is  the  fraction  of  Y-f ire  directed  at  X  . 

The  physical  interpretation  of  this  idealized  military 
situation  is  as  follows :   combat  for  a  known  length  of  time  between 
an  X  force  composed  of  two  types  of  weapon  systems  (for  example, 
infantry  riflemen  and  machine  gunners)  and  a  homogeneous  Y   force 
(for  example,  riflemen  only).   The  objective  of  the  Y-force 
commander  is  to  maximize  the  military  worth  of  his  surviving  forces 
at  the  end  of  a  battle  lasting  a  known  length  of  time  T  and  to 
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minimize  that  of  the  enemy's.   This  is  accomplished  by  his  choice 
of  the  fraction  of  fire,   0,   directed  at  the  X   forces.   A 
schematic  of  this  scenario  is  shown  below.   It  is  assumed  that  a 
military  worth  can  be  assigned  to  survivors  of  each  force-type  and 
that  this  utility  is  additive.   We  denote  this  utility  per  unit  of 
weapon  system  as   p,  q  and   r  for  the  X  ,  X   and  Y  forces, 
respectively. 


Our  state  equations,  i.e.,  the  differential  equation  con- 
straints, represent  the  attrition  process  in  this  idealized  battle, 
We  shall  refer  to  attrition  as  being  a  "square-law"  process  when 
the  casualty  rate  is  proportional  to  only  the  number  of  enemy 
firers  and  as  being  a  "linear-law"  process  when  it  is  proportional 
to  the  product  of  the  number  of  enemy  firers  and  remaining  targets. 


C-6 


For  the  problems  in  this  paper  we  assume  a  "square-law"  process. 
The  interested  reader  can  find  discussions  of  the  physical  assump- 
tions which  lead  to  this  type  of  attrition  in  articles  by  H.  Weiss 
[12]  and  H.  Brackney  [5].   The  essential  points  are  that  new  targets 
are  acquired  at  a  rate  independent  of  force  levels  and  that  fire 
is  aimed  at  point  targets.   More  recently  S.  Bonder  [3],  [4]  has 
developed  formulas  for  estimating  the  Lanchester  attrition-rate 
coefficient,  the  rate  at  which  one  unit  of  weapon  system  destroys 
enemy  targets,  as  a  synthesis  of  the  following  factors:   hit  pro- 
babilities, rates  of  fire,  target  acquisition  rate,  weapon  system 
projectile-target  lethality  characteristics,  adjustment  process. 
In  general,  the  Lanchester  attrition-rate  coefficient  is 
range  dependent,  and  the  above  formulation  (1)  takes  this  into 
account  by  having  the  attrition  coefficients  depend  upon  time.   S. 
Bonder  [2]  has  done  the  pioneering  work  on  analyzing  dynamic  combat 
situations  with  Lanchester-type  equations  and  variable  attrition 
rates  (also  see [9],  [10]  for  more  background).   For  example,  consider 
a  mobile  attack  on  a  static  defensive  position.   The  effectiveness 
(Lanchester  attrition-rate  coefficient)  of  weapons  systems  depends 
upon  force  separation,  and  this  is,  in  turn,  related  to  time  via 
the  attack  velocity.   In  [9]  we  showed  how  to  develop  Lanchester- 
type  equations  with  either  time  or  range  as  the  independent  variable 
and  noted  the  equivalence  of  the  two  formulations  (see  [10]  for 
more  extensive  results  with  variable  attrition  rates).   Hence,  the 
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above  was  our  motivation  for  studying  scenarios  with  time-dependent 
attrition  rates. 

We  finally  note  that  when  fire  is  uniformly  distributed  over 
a  target  area  [12]  or  the  rate  of  target  destruction  is  constrained 
by  the  rate  of  acquisition  of  new  targets  and  this  is  inversely 
proportional  to  target  density  [5],  a  "linear-law"  attrition  process 
results.   We  have  elsewhere  [11]  pointed  out  that  this  type  of 
attrition  structure  leads  to  a  fundamentally  different  structure 
for  the  optimal  target-type  selection  policies. 

We  now  develop  the  solution  to  the  above  problem  (1) .   In 
all  cases  we  consider  the  special  instance  when  both  X-force 
weapon  systems  are  such  that 

b1(t)  =  k^  h(t)   and  b2(t)  =  k^  h(t).  (2) 

This  physically  means  that  both  weapon  systems  have  the  same  type 
of  range  capability  (for  example,  quadratic  dependence  of  kill 
rate  on  range)  and  the  same  effective  range,  although  one  weapon 
system  dominates  the  other  in  exactly  the  same  manner  at  all  ranges. 

It  seems  appropriate  to  point  out  that  there  is  a  special 
instance  when  the  optimal  allocation  policy  takes  a  particularly 
simple  form.   This  is  when  the  Y  force  values  surviving  X-force 
types  in  direct  proportion  to  their  kill  rate  against  the  Y   force, 
i.e.,   p/q  =  kb1/kb2  =  b1(t)/b2(t)  =  b1(t=T)/b2(t=T).   The  optimal 
allocation  rule  for   0  <.   t  £  T   is  then 
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l 


{ 


1  for  a  (t)b  (t)  >  a2(t)b2(t), 
0*(t)  =  \  (3) 

0   for  a1(t)b1(t)  <  a2(t)b2(t). 


This  was  obtained  by  application  of  optimal  control  theory  [6]  as 
follows . 

The  Hamiltonian  is  given  by 

H(t,xi,pi,0)  =  -p1(t)0a1(t)y  -  p2(t)(l-0)a2(t)y 

-  p3(t){b1(t)x1  +  b2(t)x2),  (4) 

where  the  dual  variables,   p.(t)   for   i  =  1,2,3,   are  the  effect 
on  the  total  value  of  survivors  at  the  end  of  battle  when  an  optimal 
policy  is  followed  from  time   t  until  time  T  of  having  an  addi- 
tional unit  of  x.(t)   [here  we  have  let  y(t)  =  x  (t) ] .   Another 
way  to  state  this  is  that 

P±(t)  =  -^  (t)   where  S  =  SCtjX^x^y)   is 
i 

the  optimal  value  function.   According  to  the  maximum  principle, 

the  optimal  control  is  determined  by 

maximize  H(t ,x. ,p . ,0) ,  (5) 

0*0*1  X 

and  this  leads  to 

1   for  v(t)  >  0, 


{ 


0*(t)  = 

0   for  v(t)  <  0,  (6) 
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where 


v(t)  =  -p1(t)a1(t)  +  p2(t)a2(t). 


The  adjoint  system  of  differential  equations  for  the  dual  variables 
is  given  by 

dPi 


(7) 


dt      9x     1    3 


with  p1(t=T)  =  -p, 


dT=-^2   =   b2(t)p3 


with  P2(t=T)  =  q, 


dpT       3H 

dt    ~  to  =  0ai(t)Pi  +  (l-0)a2(t)p2  with  p3(t=T)  =  r. 


(8) 


By  assuraption  (2)  and  the  adjoint  system  (8)  ,  it  is  readily  shown 
that 


dp 


kb. 


i       i 


dp2   kb. 


which,  when  integrated,  leads  to 


kb. 


Px(t)  =  ^—  (p2(t)  +  q}  -  p. 


(9) 


Using  (9),  we  may  write  v(t)   defined  by  (7)  as  follows 


v(t)  = 


\. 


\. 


(-PjCtrtft,,  al(t)  -  ^  a2(t)>  +  k,  ,  {£  -  ^}   al(t) 
12  2         b_ 


(10) 
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Assuming  that   p/q  =  d  /  c>9>   we  see  that  the  optimal  target  selec- 
tion rule  as  given  by  (3)  is  an  immediate  consequence  of  (6) ,  (7) 
and  (10).   We  have  also  made  use  of  the  fact  that   p9(t)  <  0,   which 
is  easily  shown. 

We  now  consider  the  case  when 

a, (r)  =  k  h(t)   and  a0(t)  =  k  h(t) .  (11) 

1       al  2       a2 

This  means  that  all  four  attrition  rates  are  proportional  to  the 

same  basic  time  dependence.   As  we  have  pointed  out  elsewhere,  both 

the  solution  to  Lanchester-type  equations  [9]  and  the  solution  to 

the  fire  distribution  problem  [11]  in  this  instance  are  essentially 

the  same  as  for  constant  coefficients,  only  with  a  transformation 

of  the  time  scale.   The  solution  is  shown  in  Table  I. 

We  develop  the  solution  shown  in  Table  I  for  the  nonrestric- 

tive  assumption  that  k  k   >  k  k    as  follows.   Consider  the 

al^l    a2  b2 

quantity  v(t)/h(t)   which  by  (7)  is  seen  to  be  given  by 


1 
Differentiation  of  (12)  leads  to 

d  rv(th  _ 


TM  -   "Pl(t)ka,  +  "2(t)ka  •  (12) 


ttim)  -  -^\\-\\^3-  (13) 


where  we  have  made  use  of  both  the  adjoint  equations  (8)  and  also 
assumption  (2).   Observing  that  h(t)  >  0,   it  is  clear  that  the 
control  law  (6)  is  equivalent  to 
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1  for||>  0, 

(14) 
0  for  ^}  <  0, 

Since  we  develop  the  solution  to  this  problem  by  working  backwards 

from  the  end   t  =  T,   it  is  convenient  to  introduce  the  "backwards 

time"  variable  t   defined  by   x  =  T  -  t.   Observing  that  —  =  -  -j—. 

dt     dx 

we  have  that 

It  is  easily  shown  that  p~(t)  >  0   for  all   t.   Hence,  it  follows 
from  (15)  that 

d  rv(x) 


d 


xth(x7  "  °  for  a11  T'  (16) 


since  h(t)  >  0   and  we  have  assumed  that  k  k,   >  k  k,  . 

al\  a2^2 

To  solve  this  problem,  we  work  backwards  from  the  end  of 

battle  at   t  =  T,   or  equivalently   x  =  0.   At  this  point  we  have 

v(t=T)  =  h(t=T)(pk  -qk   ).  (17) 

al    a2 

Thus,  we  have  two  cases  to  consider. 

In  Case  (A):   k  p  ^  k  q,   we  have  that  v(t=T)  ;>  0  by 
al     a2 
(17) .   Consideration  of  (16)  immediately  yields  that    ,    !    >   0   for 

all   x  >  0  and  thus  0*(t)  =  1   for  0  a   t  <   T  by  (14). 

In  Case  (B) :   k  p  <  k  q,   we  have  that  v(t=T)  <  0  by 
al     a2 
(17)  and  the  assumption  that  h(t=T)  >  0   (the  battle  ends  with 

non-zero  attrition  rates).  Thus 
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h(x=0)     U' 


with 


-rSH)    >  0   for  all 
dx  ^h(x); 

v(t) 


Thus,    .    \      must  change  from  a  negative  quantity  to  a  positive  one 

at  some  time.   Let   t,   denote  the  time  when  this  happens.   Then 

for  0  £  x  <  t,   we  have  that   0*(x)  =0.   We  can  easily  determine 

p„(x)   during  this  interval  by  considering  the  adjoint  equations 

(8)  which  may  be  written  as 

dp2 

=   -b0(T)p0(x)  with     p_(t=0)    =  -q, 


dx  -2vt/t-3\l/  ™"     v2 

and 

dp3 

-^-  =  -a2(x)p2(x)  with     p3(x=0)    =   r.  (18) 

These  variable  coefficient  adjoint  equations  are  readily  integrated, 
however,  by  consideration  of  (2)  and  (11),  i.e.,   a_(x)/b  (x)  = 
constant,   and  the  observations  elaborated  upon  in  [10],  i.e., 
from  (18)  we  can  obtain  the  following  linear  second  order  differ- 
ential equation  for   P9(x) 

d2p2   a2(x) 

dvT^  "  0^x7  P2  =  °' 

with 


P2(u=0)  =  -q, 

dp2 

du1  (U=0)  =  "r> 
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and 


u  = 


b2(i)dT 


Hence,  we  readily  find  that 


P2(T)  =  -cl  cosh  e(T) 


t)  -  r  /t-^" 


2 

s  inh  0 ( t ) , 


(19) 


where 


e(x)  =  /k  K 

a2^2 


h(x)dx 


(20) 


Substituting  (19)  into  (10) ,  we  obtain  that 


v(t)    = 


h(x) 


h 


+  r  /  — 


2L 


{q   cosh   0 (x) 


kb. 


—  sinh   0(x)}(k     k     -k     k)    +  k     k,     q{£  -  r-i} 
ka2  al   bl      a2%2  al\      1        ^ 


,   (21) 


where   0(x)   is  given  above  by  (20).   Observing  that   x    is  deter- 
mined by  the  transcendental  equation  v(x  )  =  0,   we  see  that  we 
have  obtained  all  the  results  presented  in  Table  I. 
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3.   Several  Target  Types,  Constant  Attrition  Rates. 

We  consider  the  following  prescribed  duration  battle: 


n 

maximize  vy(T)  -  J  w.x.(T)    with  T   specified, 

0.(t)  i=l      X   X 

dx. 

subject   to:     - —  =  -0.a.y     for      i  =   l,...,n, 
J  dt  x   x  '         '    ' 


dt  ,L.      x   x 


i=l 

n 
x.,y  ?>  0,      0.2:0      for      i  =   l,...,n     and        £      0.    =   1, 
11  i=l      1 

where  v  and  w.   for   i  =  l,...,n  are  values  placed  on  surviving 

specific  force  types  and  all  other  symbols  are  used  in  the  same 

sense  as  above.   We  now  present  the  details  behind  the  solution 

which  we  stated  in  a  companion  paper  [11]. 

The  Hamiltonian  to  the  above  problem  is  given  by 


H(t,x.,p.,0.)  =  -y  J     a.p.UXP.  -  pn+1  ?  b.x..      (22) 

x=l  x=l 


According  to  the  maximum  principle,  the  optimal  control  (there  is 
only  one  extremal)  is  determined  by  the  (trivial) linear  program 


maximize  H(t ,x . ,p . ,0 . ) 
xx   x 

0. 

X 

n 
subject  to:    /   0.  =  1, 

i-i  1 


</>i  £  o, 
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which  in  turn  leads  to 


max 


n 

imize     /, 
<P.           i=l 

X 

a.(-pi(t))0. 
n 

bject   to: 

I     0.    =   1, 
i=l      X 

<P.  £  o. 

(23) 


By  inspection  the  solution  to  (23)  is  easily  seen  to  be 

0*<t)  -Jlj(t),  (24) 

where  6..   is  the  Kronecker  delta  and  is  equal  to   1   for   i  ■  j 
and  zero  otherwise  and  j(t)   is  the  index  such  that 


a  p    (t)    =  minimum    (a  p    ,a  p    , . . . ,a  p   ) . 
JJ  1122  nn 

* 
To  trace  the  history  of  <p .      over  time,  we  must  consider 

l 

the  adjoint  system  of  differential  equations  given  by 

dpi  3H 

dT  =   "  k.    =  Pn+lbi     W±th     Pt(t=T)    =   -w.      for      i=l,...,n, 

x 

S2^  "  -  If  =  J,  0iVi   with   "„+i(t=T>  ■ v-  . (25) 

x=l 


and 


Considering  (4) ,  it  is  easily  seen  that 


dp.   b. 
x    x 


so  that 


dp    b  ' 
n    n 


b. 

Pi(t)  =  b1  {pn(t)  +  Wn}  "  Wi*  (26) 

n 
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Substituting  (26)  into  (23) ,  we  obtain  after  some  manipulation 
that  the  optimal  control  is  determined  by 


n 


maximize 


I     c,(t)0 


0.     i=l 


l    l 


n 


subject  to:    £   0.  =  1, 


i=l 


0±  >   0, 


(27) 


where 


C-Pn(t)) 

c.(t)  =  ^ b 

l         b      11 
n 


1  + 


w.    f  b    w  "\ 


(28) 


Hence,  we  see  that  if  w.  =  kb.   for   i  =  l,...,n,   i.e.,   Y 
values  enemy  survivors  in  direct  proportion  to  their  kill  rate 
against  his  forces,  the  above  problem  is  equivalent  to 


n 


maximi 


Lze        I      0.a.b 


0.  i-1 

l 


ill 


n 


subject   to:        J>      0.    =   1, 


i-1 


0±  ^  0, 


where  we  have  made  use  of  the  easily  verified  fact  that   p  (t)  <   0 
for  all  time.   Thus  the  optimal  control  is  given  by 


0.  (t)  =  6. .   for  OitiT, 


(29) 


where  j   is  the  index  such  that 
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a.b.  =  maximum  (a.b. a  b  ). 

J  J  11      n  n 

Hence,  for  this  problem  when  one  values  enemy  survivors  in  direct 
proportion  to  their  kill  rate  against  you,  the  optimal  tactic  is 
to  concentrate  all  fire  on  a  single  target  type  until  it  is  entirely 
destroyed. 

We  now  consider  the  more  complex  case  where  one  does  not 
value  enemy  survivors  in  direct  proportion  to  their  kill  rate 
against  you.   We  shall  see  that  if  the  battle  lasts  long  enough 
there  will  be  one  or  more  switches  in  the  ranking  of  target  prior- 
ities.  As  before,  since  we  develop  the  solution  to  this  problem  by 
working  backwards  from  the  end   t  =  T,   it  is  convenient  to  intro- 
duce the  "backwards  time"  variable  t   defined  by  T  =  T  =  t.   At 
the  end  of  battle   t  =  T,   we  arrange  the  enemy  target  types  so 
that  n  is  the  index  such  that 


aw     =   a    (-p    (x=0))   =  maximum    (anw, ,...,a  w  ).  (30) 

nnnn  11  nn 

By  (23)  it  is  easily  seen  that 

0*(t=T)  =  6.  .  (31) 

l         in 

By  straightforward  continuity  arguments,  it  is  readily  seen  that 

0*(t)  =  6.    for  t  <E  [0,t..),  (32) 

l       in  i 

where  x    is  the  "backwards  time"  of  the  first  switch  in  target 
selection.   Giving  consideration  to  (32)  and  observing  that 
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-7—  =  -  ~r~,     we  see  that  for   t  6  [0,t,]   we  need  only  consider  the 
dt     dx  1  J 

following  equations  from  the  adjoint  system  (25) 

dpn 

- —  =  -b  p  ,n   with  p  (x=0)  -   -w  , 
dx      nrn+l        *n  n 


dPn+l 


=  -a  p   with  p    (T=0)  =  v,  (33) 


dx        n^n        *n+l 

and  we  recall  that  (re-writing  (26)) 

b. 

p.(x)  =  —  {p  (x)  +  w  }  -  w.   for  i  =  l,...,n.       (34) 
x     b    n      n     i 
n 

The  above  initial  value  problem  (33)  is  routinely  solved  to  yield 

yr 

p    (x)    =   -w     cosh   /a  b      x   -  v/ —  sinh   /a  b      x.         (35) 
n  n  n  n  a  n  n 

n 

We  will  now  determine  what  conditions  are  necessary  for  a 

change  in  target  selection  and  the  time  at  which  the  change  occurs, 

t, .   To  do  this  we  re-write  (27)  and  (28)  as 

n 

maximize     )      e.(x)0. 
11 
<P.  1*1 

n 

subject    to:        \     0.    =   1, 

i=l      X 

0.    ^  0,  (36) 


where 

b. 

e.(x)    =  a.w    [1  +— *-  ((-P n(x))    -w   }].  (37) 

l  11  w .  d  n  n 

l  n 

We  switch  at  the  smallest   x   for  which 
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a  .w. 
1  x 


w .  b      n        n 
1  n 


-.n(-Pn(T». 


(38) 


where  i  =  l,...,n-l  and  certain  other  conditions  (to  be  deter- 
mined presently)  are  met.   Let  k  be  the  index  of  the  target  type 
to  which  fire  is  first  shifted  in  "backwards  time."  Observe  that 
at  t  =  0,   we  have 


a .w.  <  aw  , 
r  l    n  n 


(39) 


for   i  ■  l,...,n-l,   since  the  index  n  has  been  defined  by  (30). 
Then  for  t   <  t  <  t   where  x    is  the  "backwards  time"  of  the 
second  switch  in  target  selection,  we  have  that  0.(t)  =  <S   ,   and 

1  X  K. 

thus  we  must  have  by  (36)  and  (37) 

b. 


akWk 


1  +  ^rt~  h-p(t))-  w  } 

w.  b      m       n 
k  n 


>  a  (-p  (t», 


which  may  be  rearranged  to  yield 


ak(bkWn-bnWk}  K    (akbk-anbn)("Pn(T))- 


(40) 


We  now  show  that  a  necessary  condition  for  fire  to  be  shifted 

from  target  type  n  to  target  type  k  when  one  works  backwards 

from  the  end  is  that   a.  b,  >  a  b  .   The  proof  is  as  follows.   We 

k  k    n  n 

shall  show  that   a.  b,  ri,   a  b   leads  to  a  contradiction.   First,  we 

k  k    n  n 

consider  the  special  case  when  a.  b.  =  a  b  .   In  this  case  (40) 

k  k    n  n 

reduces  to 


w  b  a. 

n  n  _  _k 

w.  b,  a  ' 

k  k  n 
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or 

aw  <  a,  w,  . 
n  n    k  k 

But  this  is  a  contradiction  to  (39)  which  must  hold  with   i  =  k. 

In  the  case  when  a  b  >  a.  b.  ,   then  using  the  fact  that   (-p  (x)) 

n  n    k  k  °  rn 

>  w   for  t  >  0,   we  may  write  (40)  as 

(b  w.  -b,  w  ) 

n  k  k  n    ,    .   NN 

a1  — - —  >  (-p  (t))  >  w  , 

k  a  b   -  a.  b.      rn        n 
n  n    k  k 

but  this  leads  to  a,  w,  >  a  w   which  is  a  contradiction  to  (39) 

k  k    n  n 

as  before. 

Thus,   a.  b,  >  a  b   and  the  switch  in  target  selection  occurs 
k  k    n  n 

at 

(bkWn"bnWk) 

*kal  -ab  =  <-P»<"l»  >°>  «» 

k  k    n  n 

so  that  a  second  necessary  condition  is 

bk   Wk 

b    w 
n    n 

In  other  words,  we  switch  to  fire  at  earlier  times  in  the  battle 

on  the  target  type  which  causes  attrition  proportionally  more  than 

the  ratio  of  values  placed  on  survivors  from  the  target  which 

yields  the  greatest  direct  return  at  the  end  of  battle. 

To  recapitulate  the  above,  the  target  to  which  fire  is  first 

shifted  (working  backwards  from  the  end  of  battle)  has  index  k 

determined  by 
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R_  =  minimum  (R.  ,...,R   n),  (42) 

*    R.>0     X      n_1 


a.b .>a  b 

11  n  n 


where 


(b.w  -b  w.) 

R,  =  a,   u  n  n  u   f°r  i=l,...,n-l.  (43) 

i    l  a.b  .  -  a  b 
li    n  n 

t"  v» 
The  time  of  switch,   t  ■   of  fire  to  the  k    target  type  is 

determined  by  the  transcendental  equation 


T,     +    V/    


a.  (b.  w  -b  w  ) 


w     cosh /a  b      in    +  v/ —  smh   /a  b      T-    =  r r —   •  (44) 

n       nnl      a         nnl    a,  b.-ab 

n  k  k    n  n 


This  is  seen  to  be  a  generalization  of  the  case  for  two  target 
types  [11] . 

The  general  pattern  of  when  and  to  which  target  types  fire 
is  shifted  as  we  work  backwards  from  the  end  of  battle  does  not 
emerge  until  we  have  considered  the  second  shift  in  target  selec- 
tion.  Since  this  is  dependent  upon  the  evolution  of  target  worth, 
we  must  further  consider  the  backwards  integration  of  the  adjoint 
system  of  differential  equations.   From  above,  we  have  that 

^(t)  =  <S±k  for  t  €  (t1,t2),  (45) 

where   x    is  the  "backwards  time"  of  the  second  switch  in  target 
selection.   Giving  consideration  to  (45),  we  see  that  for   t  £  [t_ ,T«] 
we  need  only  consider  the  following  equations  from  the  adjoint 
system  (25) 
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dpk 


-  =  "Vn+l  With  «'k<Wl)  =  "V 


dPn+l 


=  ~ai,P,,  with  P^C^O  =  V.,  (46) 


where 


di      krk        rn+lv   1'    k 


a.  (b.  w  -b  w,  ) 
k  k    n  n 


=  ^ 


V  =  /  -^  (W.2-w2)  +  v2   .  (48) 

k     b    k  n 
n 


Equation  (47)  is  merely  (43)  re-written  with  a  chnage  in  notation. 

Equation  (48)  is  readily  deduced  when  we  observe  that  according  to 

(33)  a  "square  law"  relates  the  dual  variables   p  (x)   and  p  , ,  (x) 

n  n+1 

for  0  si  x  £  x 

a  {p2(x)  -  w2}  =  b  {p2   (x)  -  v2}.  (49) 

n  n       n     n  n+1 

We  further  observe  that  all  the  dual  variables  may  be  expressed  in 
terms  of  Pk(T)   (let  n  =  k  in  (26)) 

b. 

Pi(T)  =  IT  {pk(T)  +  Wk}  "  Wi  f°r   i  "  1*'"'n'  (50) 

k 

Again,  the  equations  (46)  are  routinely  solved  to  yield  for 


t   €   [x1>x2] 


[t-t,  )    -  V,  /  —  sii 


p,  (x)    =   -W     cosh/a,b      (t-t,)    -  V   / —  sinh/a,  b      (x-x,).       (51) 

k 

Subsequent  arguments  are  now  similar  to  those  given  for  the 

first  switch  in  tactics.   Let   j   be  the  index  of  the  target  type 
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to  which  fire  is  shifted  secondly  in  "backwards  time."  Then,  it 

may  be  shown  by  similar  arguments  to  above  that  necessary  conditions 

for  fire  to  be  shifted  to  the  j     target  type  are  that 


a.b.  >  a.b.  >  a  b  ,  (52) 

j  j    k  k    n  n 


and 

b .    w. 

r1  >  -1-  (53) 

bk   wk 

However,  we  gain  more  insight  by  considering  (53)  slightly 
differently,  for  it  may  be  shown  that 

b.    b.    b 

-L  >  -±   >  -A  (54) 

w.    w,    w 
J    k    n 

It  also  seems  appropriate  to  consider  the  military  interpretation 

b. 

of  the  ratio  — .   We  recall  that 
w. 

w.  =  value  per  unit  of  X.   surviving  at   t  =  T, 

b.  =  kill  rate  per  unit  of  X.   against  Y. 
x  1 


Then 


b.      kill  rate  per  unit  of  X. 


w.   value  per  unit  of  X.   survivors' 
i         r  1 


Thus,  we  see  that  as  we  progress  backwards  from  the  end  of  battle 
that  fire  is  always  shifted  to  target  types  with  larger  ratios  of 
kill  rate  per  unit  of  weapon  system  per  unit  value  of  survivors. 
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By  a  similar  argument  as  for  the  first  shift  in  fire  using 
(42)  and  (43) ,  it  may  be  shown  that  the  target  to  which  fire  is 
shifted  secondly  (working  backwards  from  the  end  of  battle)  has 
index  j   determined  by 


S.  =  minimum  (S15...,S  ),  (55) 

J     S.>0    l  n 

a  .b  .  a.  b, 
lr  k  k 


i^k 


where 


a.  (b  .w  -b,w.) 

S.  =  r r —     for  i  =  l,...,n.         (56) 

1    aibi  "  \bk  ±k 

The  time  of  switch,   t  ,   of  fire  to  the  j     target  type  is 

determined  by  the  transcendental  equation 

, f\  

Wk  cosh/a^  (x2-r1)   +  v/ —  sinh/a^  (t^   ) 

a.  (b  .w,  -b.  w.) 

=  _J J    k     k  J       . 

a.b  .    -  a.  b. 
J    J  k  k 


(57) 

i 

Further  shifts  in  fire  follow  the  pattern  established  above. 

Based  on  our  above  development,  we  now  trace  the  course  of 
battle  forward  in  time.   We  assume  that  no  force  type  is  ever 
reduced  to  zero.   (If  thiG  does  happen,  though,  it  may  be  shown 
that  fire  is  merely  shifted  to  the  surviving  target  type  of  highest 
priority.)   Then,  we  have  that 


C-  26 


(a)   when  x   >  T 


0.(t)  =6.    for  OitiT, 
1       in 


(b)   when  x   £  T  >  x 


0.  (t)  =  6.,  for   0  £  t  £  T  -  t.  , 
i       lk  1 

0.(t)  =  6.  for  T  -  t.  £  t  £  T  -  T„, 
x       in  1  2 


(c)   when   x   £  T  >  t„ 

0. (t)  =  6. .   for  0  £  t  £  T  -  x„, 
I       xj  2 

0±(t)  =  6±k  for  T  -  x2  £  t  £  T  -  T±, 

* 
0. (t)  =  6.   for  T  -  T-  £  t  £  T, 
l       in  1 

and  so  forth.   t1   and  x„   are  determined  by  (44)  and  (57), 

respectively,  and  a  similar  expression  exists  for   x  .   The  number 

of  switches  in  target  engagement  depends  upon  the  scheduled  length 

of  battle,   T.   As  the  battle  progresses  forward  in  time,  fire  is 

b. 
always  shifted  to  a  target  type  for  which  both  a.b.   and  —  are 

i 
smaller  than  the  target  type  previously  engaged.   If  the  battle  is 

scheduled  to  last  long  enough,  then  during  the  initial  stages  of 

all  fire  is  concentrated  on  the  target  type  for  which  both  the 

b. 
quantities   a.b.   and  —  are  larger  than  any  other  target  type. 

i 
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4.   Several  Target  Types,  Some  Special  Cases  of  Variable  Attrition 
Rates . 

We  consider  the  following  prescribed  duration  battle: 


imize  {vy(T)  -  £  w.x.(T)}   with  T  specified, 


0.(t)  i=l   1  L 

dx. 
subject  to:  -7—  =  -0.a.(t)y  for   i  =  l,...,n, 


n 

I 

i-1 


&  -  - 1  b^>  v 


n 
x.,y  ;>  0,   0.  ^  0   for   i  =  l,...,n  and   £   0.  =  1, 
11  i=l   X 

where  all  symbols  are  used  in  the  same  sense  as  above.   We  shall 
omit  some  details  in  the  development  of  the  solution  to  this  pro- 
blem, since  the  development  is  a  direct  synthesis  of  details  from 
the  two  previous  problems.   In  fact,  an  extensive  elaboration  on 
the  nature  of  the  optimal  engagement  rules  will  not  be  necessary, 
since  we  shall  consider  some  special  cases  of  variable  attrition 
rates  for  which  the  solution  is  similar  to  the  constant  attrition- 
rate  case  considered  in  section  3. 

In  all  cases,  we  assume  that  the  X-force  weapon  systems 
have  performance  characteristics  sich  that 

b  (t)  =  k^  h(t)   for  i  =  l,...,n.  (58) 

i 

As  before,  we  first  consider  the  special  case  when  X-force  types 

are  valued  in  direct  proportion  to  their  kill  rate  against  the 

Y   forces  at  the  end  of  battle 


C-28 


w.  =  cb.(t=T)  =  ck.  h(t=T).  (59) 

i 

The  optimal  allocation  rule  consequently  takes  a  particularly 

simple  form  for  0  m   t  £  T 

<(t)  -  «1)j(t),  (60) 

where  j (t)   is  the  index  such  that 

a.(t)b.(t)  =  maximum  (a.  (t)b- (t)  ,  .  .  .  ,a  (t)b  (t)). 
J    3  11         n    n 

Similar  to  our  previous  developments,  the  above  solution  is 

developed  as  follows.   The  adjoint  system  of  differential  equations 

is  given  by 

dp. 

=  b.(t)p  .      with  p.(t=T)  =  -w.   for   i  =  l,...,n, 


dt     ix  /ln+l        ri'   '     i 

dp  .,     n 

~J^=     I     0ia.(t)p.   with  pn+i(t=T)=v.  (61) 

i-1 


It  is  readily  seen  that  assumption  (58)  leads  to  the  result  that 

V 

P,(t)  =  t-+   {p(t)  +  w  }  -  w  (62) 

i      R,     n       n     l 

n 

for   i  =  l,...,n.   It  may  be  shown  that  the  maximum  principle  leads 

to 

n 


maximize      )      0.c.(t) 
0.  i=l 

n 
subject    to:         £      <p .    =   1, 


1=1      X 


0±  >  0,  (63) 
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where 


(-p  (t)> 


\ 


1  + 


w.    (     b  w  ) 


(64) 


The  special  assumption  (59)  that  force  types  are  valued  in  propor- 
tion to  their  kill  rate  leads  to  simplification  of  (64)  ,  namely 

(-P(t))  (-P(t)) 

c  (t)  =  — j-S a  (t)k   =    n     a  (t)b.  (t)  , 

i        k       x         n .     b  (,t;    l    i 
d  in 

n 

whence  our  result  (60),  since  it  is  readily  shown  that  p  (t)  <  0. 

We  now  consider  the  more  complex  case  when  enemy  force 
types  are  not  valued  in  proportion  to  their  kill  rates.   Again, 
we  consider  a  special  case  when  relatively  simple  analytic  results 
are  still  possible.   Thus,  we  assume  that 

a.(t)  =  k  h(t)   for   i  =  l,...,n.  (65) 

i 

In  the  case  of  a  mobile  attack  against  a  static  defensive  position, 
assumptions  (58)  and  (65)  have  the  physical  interpretation  that 
all  weapon  systems  and  weapon  system  combinations  have  the  same 
effective  range  and  the  same  type  of  range  dependency  for  their 
kill  rate  against  any  target  type.   As  always,  we  develop  the 
solution  by  working  backwards  from  the  end  of  battle;  so  it  is 
convenient  to  introduce  the  "backwards  time"   x  =  T  -  t.   Let  n 
be  the  index  of  the  target  type  fired  on  at  the  end  of  battle. 
Then 


a  (t=0)w  =  a  (x=0)(-p  (t=0)) 
n      n    n        n 
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=  maximum  (a  (x=0)w  , .  . .  ,a  (t=0)w  ) 


(66) 


Then  from  (63)  and  (64)  for  0  £  x  <  x  ,   we  have  that 


^(T)  =  «in> 


where  x1   is  the  "backwards  time"  of  the  first  switch  in  target 
type  at  which  the  Y   forces  fire.   As  before,  the  above  "bang- 
bang"  optimal  control  leads  to  simplification  of  the  adjoint  system 
of  differential  equations,  and  it  readily  follows  that 


p  (x)  =  -w  cosh  6  (x) 
n        n       n 


x)  -  v/  - 


sinh  9  (x) , 
k         n 


n 


(67) 


where 


n 


(x)  -  /TT^ 


h(x)dx 


n  n 


0 


(68) 


In  deriving  the  above  we  have  used  the  same  results  for  variable 

coefficient  differential  equations  used  in  section  2  and  noted 

in  [9]. 

Let  k  be  the  index  of  the  target  type  to  which  fire  is 

first  shifted  in  "backwards  time."  Then  for   t.  <  x  <  x9  we  have 

that  0.(x)  =  6..  .   To  determine  the  index  k,   we  re-write  (63) 
i      Ik 

and  (64)  as 


n 

maximize      )      0.e.(x) 

0.  1=1 

1 

n 
subject   to:        \     0.    =   1, 
i=l      1 
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0.    ^  0, 

x 


(69) 


where     e.(x)      is  given  by 


e.  (x)    =  a.  (t)w. 


kb. 


1  +  — rM(-p    (t))    -  w   \ 

w.k,     I       n  n; 

lb 


(70) 


The  condition  that   en  (t)  >  e  (x)   for  x  >  xn   leads  to 

k       n  i 


k   k     n         k  k   n  n 


(71) 


which,  in  turn,  yields  that  necessary  conditions  for  fire  to  be 
shifted  as  we  work  backwards  from  the  end  to  the  k    target  type 


are 


c  k   >  k  k 
a.  d.     a  b 
k  k     n  n 


(72) 


and 


— £>— a 

W.      W 

k     n 


(73) 


Thus ,  it  follows  that  the  index  k  is  determined  by 


R,     =  minimum    (R    ,  . 

k  R.>0  l 

k     k  Sk     k 
a.   b .      a     b 
li       n     n 


-W- 


(74) 


where 
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ka.(kb.Wn_kb  wi> 


R.  =  .  \     X  .  ? •  (75) 

x   k  k   -  k  k 

a.  b .  a  b 
11     n  n 

The  "backwards  time"  of  switch,   t  ,   of  fire  to  the  k    target 
type  is  seen  to  be  given  by  the  transcendental  equation 


/x 

jj   +  v/  -. sinh  6    (t 


w     cosh  6    (t.)   +  v/  — -  sinh  6    (tJ    =  R.  ,  (76) 

n  n     1  k  n     1  k. 

a 
n 

where  6  (t)   is  given  by  (68).   This  is  seen  to  be  a  generalization 
of  corresponding  results  in  the  two  previous  sections. 

Further  details  and  results  are  similar  to  those  presented 
in  section  3.   However,  they  differ  in  the  same  fashion  as  the 
earlier  developments  in  this  section  differ  from  corresponding  ones 
in  section  3.   We  finally  observe  that  the  optimal  target  engage- 
ment policies  are  the  same  as  those  given  at  the  very  end  of 
section  3.   Changes  in  target  priority  are  similarly  determined 
(compare  (42)  and  (43)  with  (74)  and  (75)).   As  noted  elsewhere 
[9],  [11],  however,  the  time  scale  of  battle  has  been  transformed 
as  evidenced  by  a  comparison  of  values  for   t,   as  determined  by 
(44)  with  (76). 
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5.   Comments . 

In  a  companion  paper  [11],  the  interested  reader  can  find 
an  extensive  discussion  of  the  structure  of  the  optimal  target 
engagement  policies  for  the  problems  whose  solutions  have  been 
developed  in  this  paper.   We  have  contrasted  there  the  structures 
of  the  optimal  allocation  policies  for  both  these  problems  and 
also  other  tactical  allocation  problems.   Here,  we  will  make  some 
further  comments,  however,  about  the  optimal  target  engagement 
policies. 

Let  us  first  note,  however,  that  for  the  problems  considered 
in  this  paper  the  optimal  control  was  always  "bang-bang,"  i.e., 
an  extreme  point  of  the  constrained  control  variable  space.   Thus, 
we  saw  that  the  optimal  tactic  was  always  to  concentrate  all  fire 
on  the  appropriate  target  type.   It  may  be  shown  that  there  are 
no  singular  extremals  (see  pp.  246-247  of  [6])  on  which  the  optimal 
control  is  an  interior  point  of  the  control  variable  space.   This 
happens,  for  example  in  the  problem  of  section  2,  because  although 

the  Hamiltonian  is  a  linear  function  of  the  control  variable,  the 

9H 
coefficient  of  the  control  variable,  i.e.,  — ,   cannot  vanish  over 


a  finite  interval  of  time.   In  other  words,  it  is  impossible  that 

dtl90; 

Let  us  now  consider  the  most  general  case  of  target  selection 

in  combat  against  heterogeneous  forces,  the  problem  of  section  4. 

We  did  not  include  this  problem  in  a  companion  paper  for  reasons 
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of  brevity.   For  this  problem,  our  basic  assumption  was  that 

b.(t)  =  k  h(t).   Then,  when  survivors  were  valued  in  proportion 

i 
to  their  effectiveness  (i.e.,  kill  rate  against  the  Y   forces), 

the  optimal  tactic  took  a  remarkably  simple  form:   always  concen- 
trate all  fire  on  the  target  type  for  which  a.(t)b.(t)   is  the 
largest. 

To  obtain  simple  analytic  results  when  X-force  types  were 
not  valued  in  proportion  to  their  kill  rate,  we  further  assumed 
that   a.(t)  =  k  h(t).   Then  as  the  battle  developed  forwards  in 

1  3.  , 

i 

time,  the  concentration  of  all  fire  on  one  target  type  was  always 

shifted  to  a  new  target  type  for  which  both  quantities   a.(t)b.(t) 
b.(t)  X    X 

and  were  smaller  than  those  corresponding  to  the  previously 

w . 

l 

engaged  target  type. 

In  this  paper  we  have  attempted  to  show  how  some  seemingly 
complex  optimal  control  problems  may  readily  yield  relatively  simple 
analytic  solutions  when  special  mathematical  structures  of  the  pro- 
blem are  exploited.   We  first  pointed  out  such  possibilities  in  a 
recent  note  [9]  (where  we  inadvertently  rediscovered  some  results 
apparently  first  observed  by  R.  Isaacs).   Although  the  solution  to 
matrix  differential  equations  (such  as  those  encountered  in  sections 
3  and  4)  is,  in  general,  messy  at  best  when  specific  solutions  are 
required,  the  occurrence  of  a  "bang-bang"  optimal  control  in  these 
optimization  problems  reduced  the  complexity  of  the  solution  appre- 
ciably.  We  have  purposely  presented  a  sequence  of  problems  to 
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show  the  reader  how  we  set  about  to  exploit  such  structural  pro- 
perties.  Our  personal  opinion  is  that  this  could  be  done  more 
frequently  in  applied  areas  like  operations  research  where  phenomena 
(here  military  operations)  is  to  be  studied. 
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6.   Summary. 

In  this  appendix  we  have  presented  the  solution  details  for 
two  problems  discussed  in  a  companion  paper  [11]  and  then  consid- 
ered a  more  general  problem.   We  have  shown  how  optimal  controls 
may  be  synthesized  in  a  simple  fashion  when  special  mathematical 
structures  of  the  problems  at  hand  are  exploited.   By  considering 
a  sequence  of  simplified,  yet  increasingly  complex  specific  pro- 
blems, we  have  hopefully  laid  the  groundwork  for  studying  more 
general  and  complex  structures  in  the  optimization  of  combat 
dynamics. 
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Appendix  D.   Target  Selection  in  Lanchester  Combat:   Linear -Law 
Attrition  Process. 

1 .   Introduction. 

In  this  appendix  we  develop  the  solution  to  a  simple  problem 
of  target  selection  in  Lanchester  combat  against  two  enemy  force 
types  which  undergo  "linear-law"  attrition.   In  addition  to  the 
Pontryagin  maximum  principle,  the  theory  of  singular  extremals 
is  required  to  solve  this  problem.   Our  major  contribution  is  to 
show  how  to  synthesize  the  optimal  target  selection  policies  from 
the  basic  optimality  conditions.   This  solution  synthesis  method- 
ology is  applicable  to  more  general  dynamic  (tactical)  allocation 
problems.   For  constant  attrition  rates  we  show  that  whether  or 
not  changes  can  occur  in  target  priorities  depends  solely  on  how 
survivors  are  valued  and  is  independent  of  the  type  of  attrition 
process . 

In  Appendix  B  we  have  presented  some  elements  of  a  mathe- 
matical theory  of  target  selection  in  dynamic  combat  situations. 
We  did  this  through  the  examination  of  the  structure  of  the 
optimal  allocation  policies  for  some  tactical  situations  described 
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by  Lanchester-type  equations  of  warfare.   The  purpose  of  this 
previous  appendix  was  to  contrast  the  structures  of  the  optimal 
allocation  policies  for  various  scenarios  without  cluttering  the 
discourse  with  the  mathematical  details  of  solution  development. 
In  the  present  appendix  we  develop  results  for  a  prescribed  dura- 
tion battle  in  which  enemy  target  types  undergo  a  "linear-law" 
attrition  process  (see  section  3  below) .   We  had  previously  [8] 
just  stated  these  results  without  justification. 

The  problem  under  study  is  solved  by  the  mathematical 
theory  of  optimal  control.   Its  solution,  however,  requires  more 
than  the  well-known  Pontryagin  maximum  principle  [7]:   the  theory 
of  singular  extremals  (see  Chapter  8  of  [2])  must  be  used  to  solve 
it.   A  brief  discussion  of  the  required  theory  of  singular  extremals 
is  included  in  this  appendix.   By  an  extremal  we  mean  a  battle 
trajectory  on  which  the  necessary  conditions  for  optimality  are 
almost  everywhere  in  time  satisfied. 

The  major  contribution  of  this  appendix  is  to  show  how  to 
synthesize  the  optimal  control  in  combat  against  target  types  which 
undergo  a  "linear-law"  attrition  process.   In  this  case,  singular 
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subarcs  (see  section  2.  below)  may  be  present  in  the  battle  trajec- 
tory.  By  the  synthesis  of  optimal  control,  we  mean  the  explicit 
determination  of  the  time  history  of  the  optimal  control  from 
initial  to  terminal  time  as  a  function  of  the  initial  state  of 
the  system.   There  is  no  general  method  for  the  synthesis  of  optimal 
controls  in  singular  problems  [6] ;  each  class  of  problems  possesses 
its  own  peculiarities.   Hence,  an  understanding  of  how  to  synthesize 
the  optimal  control  in  this  elementary  problem  is  particularly 
important,  since  it  provides  insight  for  more  complex  extensions 
that  we  have  considered  in  our  subsequent  researches. 

The  body  of  this  appendix  is  organized  in  the  following  fashion, 
First,  we  review  that  part  of  the  theory  of  singular  extremals 
which  is  required  for  the  solution  of  the  problem  under  study. 
Next,  we  present  our  model  and  develop  the  basic  necessary  condi- 
tions of  optimality.   Then,  we  show  how  to  synthesize  the  solution 
to  our  problem.   This  is  done  for  the  two  cases  of  import.   Finally, 
we  make  some  comments  about  the  structure  of  the  optimal  target 
selection  policies  and  extensions. 

2.    The  Theory  of  Singular  Extremals. 

In  an  optimal  control  problem,  the  maximum  principle  may  fail 
to  determine  an  optimal  trajectory,  since  the  maximization  of  the 
Hamiltonian  may  not  lead  to  a  well-defined  expression  for  optimal 
control  [5],  [4]  (also  see  Chapter  8  of  [2]).   Singular  solutions 
usually  occur  when  the  Hamiltonian  is  a  linear  function  of  the 
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control  variables.   However,  all  problems  for  which  the  Hamiltonian 
is  a  linear  function  of  the  control  variables  do  not  have  singular 
subarcs  in  their  solution. 

The  problem  that  we  shall  consider  has  one  control  variable, 
and  it  appears  linearly  in  the  Hamiltonian.   By  a  singular  sub  arc 
we  denote  that  part  of  an  optimal  trajectory  on  which  the  maximum 
principle  cannot  be  used  to  determine  the  control  because  the 
coefficient  of  the  control  variable  in  the  Hamiltonian  is  zero  (see 
pp.  226-227  of  [4]).   Then  the  term  "singular  soltuion"  will  be  used 
to  refer  to  any  optimal  trajectory  which  contains  one  or  more 
singular  subarcs. 

To  elaborate  further,  when  the  Hamiltonian  H  is  a  linear 

3H 
function  of  the  control  variable  0,  then  if  ttt  =0  for  a  finite 

interval  of  time  (or,  another  way  to  say  this,  the  coefficient  of 

0  vanishes  identically  for  a  finite  interval  of  time) ,  then  the 

maximum  principle  does  not  determine  the  control.   Observe  that  in 

this  case  all  feasible  values  of  0  maximize  the  Hamiltonian.   When 

this  happens  we  determine  the  singular  control  by  requiring  that 

SH  SH 

we  remain  on  the  singular  subarc,  i.e.  -jt-t-  remains  zero.   If  -zrr   is 

to  be  identically  equal  to  zero  for  a  finite  interval  of  time,  then 

all  its  derivatives  with  respect  to  time  must  also  be  equal  to  zero. 

We  determine  the  singular  control,  which  keeps  the  system  on  the 

singular  subarc,  by  considering  as  many  of  the  time  derivatives  of 

3H 

—  as  are  required  for  the  control  variable  0  to  appear  explicitly 

30 
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so    that  it  may  be   determined  from  an  algebraic  equation.      Thus, 
in  general  we    consider 

0  =  ill  =  JL/M)    =  AA    m  (1) 

dt 

2 
d   3H 
For  the  problem  at  hand,  the  equation  — ^("^7*)  =  0  and  the  canonical 

dt2  ^ 
equations  (i.e.  both  state  and  adjoint  system)  lead  to  an  explicit 
expression  for  the  singular  control. 

We  must  further  check  to  make  sure  that  we  can  get  a  maximum 
return  (in  the  case  when  we  wish  to  maximize  the  criterion  functional) 
from  use  of  the  candidate  singular  subarc.   The  following  condition 
(generalized  Legendre-Clebsch  condition)  is  necessary  for  a  singular 
subarc  to  yield  a  maximum  return 

2k 
dt 
It  is  obtained  by  examining  the  negative  semidefiniteness  of  the 
second  variation  for  a  special  class  of  explicitly  defined  control 
variations  [6].   For  the  problem  at  hand,  it  suffices  to  consider 
the  generalized  Legendre-Clebsch  condition  with  k  =  1.   Recently, 
Jacobson  [3]  has  discovered  a  new  necessary  condition  for  optimality 
on  singular  subarcs.   This  condition  is  not  readily  checked,  how- 
ever, for  the  problem  at  hand,  since  the  details  of  application  are 
extremely  messy. 
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3.   The  Model  and  Development  of  Basic  Optimality  Conditions. 
We  consider  the  following  prescribed  duration  battle 

maximize  {ry(T)  -  px  (T)  -  qx  (T)}   with  T   specified, 
Ht) 

dXl 
subject  to:   -7— -  =  -Pa.x.y 
J  dt       11 

dx2 

IT  =  -(H)a2¥' 

ifc  =  -Vi  -  v2, 

x1,x2,y  £  0  and  0  £  9>  *.   1,  (3) 

where 

p,  q  and   r  are  values  placed  on  surviving  force  types, 
x..  ,  x_   and  y  are  average  force  strengths, 
a..  ,  a? ,  b1   and  b_   are  constant  attrition  rates, 
(observe  that  the  a's   and  b's   are  different  in  nature), 
0  is  the  fraction  of  Y-f ire  directed  X  . 

In  previous  papers  [8] ,  [9]  we  have  described  the  basic  scenario 
under  consideration  and  also  the  circumstances  which  lead  to  a  "linear- 
law"  attrition  process.   As  before  [8],  we  refer  to  attrition  as 
being  a  "linear-law"  process  when  the  casualty  rate  is  proportional 
to  the  product  of  the  number  of  enemy  firers  and  remaining  targets. 
We  have  also  discussed  at  length  the  structure  of  the  optimal  target 
selection  policies  and  its  implications  for  military  tactics 
previously  [8]  . 
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We  now   develop   the  basic  optimality   conditions  which  hold   on 
extremals.      The  Hamiltonian   for   the   above  problem  is   given  by 
U(t,x±tp±,0)    =    (-p1a1x1y  +  p2a2x2y)0 

+   {"P2a2x2y   "  P3(blXl  +  b2x2)}'  ^ 

where   p.  (t)    for   i  =   1,2,3  are   the   dual  variables   corresponding   to 

the   state  variables   x    ,x„,x„   =  y    (see    [8],    [9]    for   a  discussion  of 

the  military   significance   of   these  variables).      The  maximum  principle 

leads    to   the    following  non-singular  optimal   control    (there   is   only 

one  extremal) 

r 


* 

0  (t)  = 


0   for  p2a2x2  <  p^^, 
[  1   for  p2a  x  >  p-a.x- . 


(5) 


The  adjoint  system  of  differential  equations  for  the  dual  variables 
is  given  by 

dp1 

—  =  0a]yp1  +  blP3  with  p][(t  =  T)    =   -p , 

dp2 

tj£-  =    (1   -  0)a2yp2  +  b2p3  with   p2(t   =  T)    =   -q, 

dp  3 

^— -  •  0a1x1p1  +    (1  -  0)a2x2p2       with  p3(t  =  T)    =  r.      (6) 

Conditions    (1)    for   a  singular  subarc  yield    for   the  problem 
at  hand 

PlalXl  =  P2a2X2>  (7) 


and 


alblXl  =  a2b2x2"  ^ 
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Differentiation  of  (8)  and  combination  with  the  state  equations 

(3)  yields  the  singular  control  required  to  remain  on  a  singular 

sub  arc 

*     a2 

0  =  a-fr--  (9) 

al  +  a2 
Checking   the   generalized  Legendre-Clebsch  condition    for   this   singular 
subarc,   we    find   after  a  rather   laborious    computation  which   requires 
use   of  both    (3)    and    (6)    that 

^^T(lf)}   =  y2P3(t){Ca1)2b1x1  +    (a2)2b2x2)    >  0, 
dt 

since   it   is   readily  shown   that  p~(t)    >  0   for  all   t.      Hence,   the 

necessary   condition   is  met   for   the   singular  path   to  be  optimal. 

In   synthesizing  the  optimal   course   of  battle    (backwards   from 

the   end  of   the  prescribed  duration  battle)    it   is    convenient   to 

introduce 

v(t)  =  _a1P1x1  +  a2p2x2<  (10) 

By  (5)  and  (9)  the  optimal  control  may  be  expressed  in  terms  of 
v(t)  as 

1         for  v(t)  >  0, 

At)  =  ^  — | for  v(t)  =  0, 

al  +  a2 

0         for  v(t)  <  0.  (11) 

We  recall  that  (8)  must  also  hold  on  a  singular  subarc. 

Since  we  develop  the  solution  to  this  problem  by  working 
backwards  from  the  end  t  =  T,  it  is  convenient  to  introduce  the 
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"backwards    time"  variable  T    defined  by  T   =  T   -   t.      Observing  that 

A  A 

—  = -  -pr  and  using  both  the  state  equations  (3)  and  the  adjoint 
system  (6) ,  we  obtain  from  differentiation  of  (10)  that 

dv 


dx   =    (aiblxl   ~  a2b2X2)p3* 


(12) 


Thus,  we  see  that  on  a  singular  subarc  on  which  v(T)  =  0  we  also 
have  that  -==■  =  0.   Also,  it  is  sometimes  convenient  to  write  (10) 
as 


v(T)  =  - 


f 

Pl^l 

(a^bjXj)  - 

a2b2x2 

lb2      J 

P20 

fbr 

ib2> 

), 

(13) 


At   the  end  of  battle   t  =  T,   we  have 

v(T   =  0)    =      a1px1(t   =  T)    -  a2qx2(t  =  T)  .  (14) 

Taking    (13)    into   consideration,  we   see    that  a  point  on   the   singular 
"surface"  a  b   x-    =   a«b-x_   yields   a  positive,    zero,    or  negative  value 
for  v(t)    at  t  =  0   depending  upon  whether  *-  is    greater   than,   equal   to, 

bl 
or  less   than  — .      Hence,  by    (11)    a  battle    trajectory  which  has   reached 

b2 
the   singular  surface   can,    in  general,    only   remain   on  it   at   the  end  of 

bl 
battle  when  *-  =  r— .      Thus,   in   synthesizing   optimal   trajectories  we 


must   consider   three   cases. 


Case    (a) 


bi 
P-  =  -A 

q  "  V 


Case    (b)        £  >  r^, 
q       b2 


L)-10 


Case  (c)   £  <  — . 
q   b0 


The  solution  for  Cases  (a)  and  (b)  has  been  described  by  us  in  a 

previous  paper  [8]. 

If  we  were  to  plot  in  Figure  1  the  line  L'  defined  by 

a1px1  =  a.qx.,  then  it  would  appear  above,  on,  or  below  the  line  L 

defined  by  a,b  x..  =  a  b~x„  depending  on  whether  ^  were  greater  than 

'  bi 
equal  to,  or  less  than  — .   This  is  evident  from  considering  the 

2 
slopes  of  these  two  lines. 


-dx^ 

aibl 

'dx„^ 

31P 

dxn 
^     1J 

L 

a2b2' 

,dxlJ 

L' 

a2q 

since,  for  example. 


dx^ 

dx,  _ 

> 

L' 

'dx2~ 

ldx1J 

bl 

*-  >  • —  implies  that 

q  b2 


The  significance  of  the  line  L'  and  its  relationship  to  the 

line  L  is  as  follows.   The  battle  is  divided  into  two  time  phases: 

Phase  I  for  0  <  t  <  t.  =T-T  and  Phase  II  for  T-T=t£t£T. 

During  Phase  I  the  optimal  target  engagement  policy  at  a  point  in 

time  is  determined  by  the  location  of  the  point  on  the  battle 

trajectory  with  respect  to  the  line  L,  which  is  also  the  singular 

"surfaces."  Above  L,  0  (t)  =  0;  while  below  L,  <P   (t)  =  1.   When 

a  battle  trajectory  reaches  L,  it  remains  on  the  singular  surface 

*     a2 
through  use  of  the  singular  control  0     = 


al  +  a2 


During  Phase  II 
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* 

the  optimal  target  engagement  policy  is  to  use  0  (t)  =  1  below 

L1.   It  may  be  shown  that  it  is  impossible  for  a  battle  trajectory 
to  cross  L'   during  Phase  II. 

The  above  results  will  be  developed  in  the  next  two  sections 
on  the  synthesis  of  optimal  control.  The  following  relationships 
readily  follow  from  previously  developed  results  and  are  required 
to  establish  the  results  of  the  above  paragraph 


v(t=0) 


(15) 


so  that 


{1   for  P(T)   below  L', 
0  for  P(T)   above  L',  (16) 

where  P(t=T)  =  (x  (t=T) ,x  (t=T) )  .   We  also  note  that  by  (12) 

/  >  0  below  L, 
^  (x)<  =  0    on   L, 

^  <  0  above  L.  (17) 

4.   Solution  Synthesis  When  Survivors  Valued  in  Proportion  to  Kill 

Rates . 

b1 
For  Case  (a):  •*-  =  — ,   optimal  battle  trajectories  are  shown 

in  Figure  1.   Above  the  line  L  with  equation  a.b  x  =  a-b-x.   the 
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* 

optimal  control  is  to  use  0  (t)  =  0  until  this  line  is  encoun- 

* 

tered.   When  a  trajectory  reaches  L,   the  singular  control   0  = 

a2 
— (which  keeps  the  trajectory  on  L)  is  used  until  the  end  of 

al    2 

battle  at   t  =  T.   Below  L,   0  (t)  =1   is  used  in  a  similar  fashion. 

To  establish  these  results,  we  work  backwards  from  each  possible 

type  of  end  point  of  battle. 

At  the  end  of  battle  t  =  0  equation  (13)  reduces  to 

v(  =0)  =  (^-}[a1b1x1(t=T)  -  a2b2x2(t=T)],         (18) 

bx 

since  we  have  assumed  -*-  =  t— .   By  (18)  we  see  that  there  are  three 

q   t>2 

cases  to  consider  depending  on  the  sign  of  the  term  in  square 
brackets. 


Case  (1)    a  b  x  (t=T)  =  a2b2x2(t=T) 

This  corresponds  to  when  the  system  ends  up  on  the  singular 

subarc.   In  this  case  0  (t=T)  =  a_/(a1+a9),   and  for  0  £  x  £  x,  = 

the  "backwards  time"  of  the  first  switch,  we  use  the  singular 

control  0  (t)  =  a9/(a  +a  ).   Let  us  note  that  use  of  the  singular 

control  for  0  £  x  £  xn   results  in  —  =0   so  that  v(x)  =  v(x=0) 

1  dx 

-x    , 

+         -=—  dx   =  0.      At      t,    =  T   -  xn      we  switch  control,    since     x1  (t.. ) 
JQ  dx  11  11 


x°   or  x_(t  )  =  x°.   This  yields  three  further  subcases. 


Subcase  (1A)    a-b-xf  <  a  b_x° 

At   t  =  t1  >  0  we  have  that  a  b  x°  =  a„b  x  (tn)  <  a  b  x°   so 
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that  we  cannot  destroy  anymore  x  .   Then  we  use  0  (t)  =  0   for 
t..  £  t  £  T.   This  is  consistent  since  v(t=t  )  =  0  and 

-j—  =  p3(a  b  x°-a  b  x2)  <  0  for  t.  £  x  £  T. 

dx2 
(Observe  that  for  t   <  t  <   T ,  -: —  =  a?x?y  so  tnat  x9(t)  >  x9(t,).) 

* 
This  implies  that  v(t)  <  0,   and  hence  0  (t)  =  0   for  0  £  t  £  t.. 

Subcase  (IB)      aibixi  >   a?b2X2 

A  similar  argument  readily  yields  that  0  (t)  =  1   for 
0  £  t  £  t  . 


Subcase  (1C)      a -.b..x°  =  a9b~x° 

* 
We  use  0  (t)  =  a  /(a  +a  )   from  the  beginning. 

Case  (2)     a^x^t^T)  <  a2b2x2(t=T) 

Since  v(t=0)  =  (-^-)  [a  b  x  -  a  b  x  ]  <  0,   at  the  end  of  battle 

X-       X   X   J-        ^-  £.      £» 

*  l 

we  have  0  (t=T)  =  0.   Hence,  for  0  £  x  £  t1  =  the  "backwards  time" 

* 
of  the  first  switch,  we  use  0  (r)  =0.   We  work  backwards  from  the 

end.   Since  we  are  above  the  line  L,  —  =  p  (a  b  x  -a_b  x  )  <  0. 

Hence,   v(t)  <  0   for  all   t  £  [0,T],   and  we  never  do  switch.   Thus, 

we  have  that  0  (t)  =  0   for  0  i  t  i  T. 


Case  (3)     a^x^t-I)  >  a2b2x2(t=T) 

A  similar  argument  to  that  used  for  Case  (2)  readily  yields  that 

* 

0  (t)  =  1   for  0  £  t  *  T. 
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The  above  cases  are  shown  in  Figure  1.   It  should  be  noted  that 
the  above  development  depends  upon  the  easily  proven  fact  that  p.(t)  >  0 
for  all  t.   It  should  further  be  noted  that,  in  general,  trajectories 
(1A) ,  (IB)  and  (1C)  will  not  all  terminate  in  the  same  point  as  shown 
in  Figure  1,  which  was  drawn  this  way  for  simplicity. 


5.   Solution  Synthesis  when  Survivors  Not  Valued  in  Proportion  to  Kill 
Rates. 


We  now  consider  Case  (b) :  *f  >  — .  Again,  we  work  backwards  from 


b 
q   b2 
each  possible  type  of  end  point  of  battle.   There  are  two  cases  to  be 

considered. 


Case  (1) :   never  on  singular  subarc  for  finite  interval  of  time. 
Again  there  are  two  subcases  to  consider,  depending  upon  whether 
the  system  winds  up  above  or  below  L. 


Subcase    (la) 


Since 


alblXl(t=T)    *  a2b2x2(t=T) 


v(x)    =  a  b1x1 


r-p2i 


(P1/P2>   a2b2x2 


(b1/b2)   a^;!*! 


we  see  that  v(t=0)  >  0   and  hence  by  (11)   0  (=T)  =  1.   Hence  for 

0  i  t  $;  t  =  the  "backwards  time"  of  the  first  switch,  we  use 

0    (t)  =1.   We  work  backwards  from  the  end  using  this  control.   Since 


df=  p3(alblXl"a2b2x2)  >   ° 
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when  we  are  below  L  and  we  stay  there  by  using  0  (t)  =  1,   we  have 
that  v(t)  >  0   for  all   t  £  [0,T],   and  hence  we  never  switch.   Thus, 
0*(t)  =1   for  0  £  t  £  T. 


Subcase  (lb)      a^  x  (t=T)  <  a2b2x2(t=T) 

Again  there  are  two  further  subcases  to  consider,  depending  upon 
whether  the  system  winds  up  above  or  below  L'. 


Subcase  (lbl)     a  b  x  (t=T)  <  a  b  x  (t=T)   and 

a1px1(t=T)  <  a2qx2(t=T) 

* 
In  this  case  we  wind  up  above  L   and  hence  by  (16)   0  (t=T)  =  0, 

Since  we  are  above  L,  —  <  0   for  all   x  by  (17).   Combining  this 

with  (15),  it  readily  follows  that  v(t)  <  0   for  all  T  €  [0,T]. 

Thus,   0  (t)  =  0   for  0  £  t  £  T. 


Subcase  (lbll)    a  b  x  (t=T)  <  a  b  x  (t=T)   and 

alPX;L(t=T)  >  a2qx2(t=T) 

In  this  case  we  wind  up  below  L'   at  the  end.   By  (15)  and  (16), 
we  have  that  v(t=0)  >  0  and  0  (t=0)  =  1.   We  work  backwards  from  the 
end.   Since  we  are  above  L,  -r-   <  0  by  (17)  while  we  remain  above 
L.   Thus  v(t)   decreases  as   x   increases.   There  are  two  further 
subcases  depending  upon  whether  v(t)   decreases  to  zero  before  the 
line  L   is  encountered.   Let   x,   be  such  that  v(x,)  =  0.   If  L 
has  not  been  reached  at   x,  ,   then  v(x)   for   x  >  x..   is  negative  and 
0  (x)  =  0   for   t  s:  x  £  T.   It  is  also  possible  to  just  reach  L 
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when  v(t  )  =  0.   In  this  case  (assuming  that  we  don't  remain  on  the 

singular  subarc)   v(t)  >  0   for  t  >  T. ,   since  we  pass  below  L  and 

then  p-   >  0. 
dt 

Case  (2) :   on  singular  subarc  for  finite  interval  of  time 
Considering  (15)  and  (17) ,  it  is  readily  seen  that  this  can  only 
happen  when  a  b  x  (t=T)  <  a  b  x  (t=T)   and  a  px  (t=T)  >  a  qx  (t=T) . 
As  usual,  we  work  backwards  from  the  end  of  battle.   By  previous 
arguments  it  is  readily  seen  that  we  use  0  (x)  =  1   for  0  £  T  £  T_ , 
and  at   t  =  t   we  must  have  a  b  x  (t  )  =  a  b  x  (t  ) .   We  use  the 
singular  control  0  (t)  =  a  /(a  +a„)   for   t  £  t  aC  t  .   There  are 
three  further  subcases. 


(1)  x1d2)   =  x°        ,       x2d2)   <  x°    , 

(2)  x1(t2)    <   x°         ,        x2(t2)    =  x°    , 

(3)  x1(t2)    =  x°         ,        X2(T2^    =  x2    ' 

We  omit  the  trivial  discussion  of  these  cases. 

Thus  we  see  from  the  above  that  there  are  six  possible  cases  for 
the  history  of  combatant  force  strengths  in  this  prescribed  duration 
battle: 

(1)  started  below  L  and  never  reached  L, 

(2)  always  above  L ' , 

(3)  started  above  L?   and  end  up  above  L  but  below  L ' 
without  ever  reaching  L, 
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(4)  end  up  above  L  but  started  below  L  and  did  not  remain 
on  L  for  finite  interval  of  time, 

(5)  started  above  (or  on)   L  and  were  on  L   for  finite  interval 
of  time, 

(6)  started  below  L  and  were  on  L  for  finite  interval  of 
time. 

bi   . 


These  six  cases  are  shown  in  Figure  2.   Case  (c)  :  •*-  <  : —  is 

q    2 
similar  to  Case  (b) . 

6.   Comments . 

Elsewhere  [8]  we  have  contrasted  the  structure  of  the  optimal 
target  engagement  policies  in  Lanchester  combat  when  the  engaged  tar- 
get types  undergo  a  "linear-law"  attrition  process  with  that  for  other 
tactical  scenarios.   An  important  question  to  be  answered  in  such  studies 
is  whether  target  priorities  change  over  time.   We  have  discovered  that 
for  the  scenarios  which  we  have  so  far  studied  the  answer  to  this 
question  is  determined  solely  by  whether  or  not  surviving  target 
types  are  valued  in  direct  proportion  to  their  kill-rate  capabilities. 
For  the  case  of  constant  attrition  rates,  changes  in  target  priorities 
over  time  can  only  occur  when  survivors  are  valued  in  excess  of  their 
kill-rate  capabilities.   This  is  true  when  the  engaged  target  types 
are  undergoing  either  a  "linear-law"  attrition  process  or  a  "square- 
law"  one  (see  [9]  for  a  discussion  of  the  "square-law"  case). 

We  now  discuss  how  the  above  principle  applies  to  the  problem 
at  hand.   When  a  linear  utility  is  assigned  to  enemy  survivors  at  the 
end  of  battle  in  direct  proportion  to  their  kill-rate  (effectiveness) 
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against  friendly  forces,  then  the  optimal  target  selection  policy 
depends  only  upon  the  location  of  the  battle  trajectory  with  respect 
to  the  singular  "surface"  L   (see  Figure  1).   Thus,  target  priorities 
don't  change  over  time  (they  can  become  equal,  however).   When  one 
target  type  is  assigned  utility  in  excess  of  its  effectiveness  (i.e., 
p/q  >  b1/b_),   then  at  time   t..   there  will  be  a  switch  from  tactics 
being  determined  by  the  location  of  the  battle  trajectory  with  respect 
to  the  singular  "surface"  L   to  being  determined  by  location  with 
respect  to  the  line  L'   (see  Figure  2).   It  may  be  shown  that   t. 
depends  on  the  particular  battle  trajectory  under  consideration  and 
no  trajectory  can  "penetrate"  L'. 

The  methodology  for  solution  synthesis  developed  in  this  paper 
is  applicable  to  more  complex  tactical  situations  of  greater  military 
significance.   Our  work  here  lays  the  foundations  for  the  study  of 
the  optimal  allocation  of  supporting  weapon  systems  (e.g.,  artillery, 
tactical  air  support,  etc.)  against  "area  targets"  (e.g.,  troop 
concentrations).   Typical  questions  of  interest  to  be  answered  are, 
"Considering  several  infantry  companies  individually  engaging  enemy 
units  of  like  size,  what  is  the  'best'  utilization  of  supporting 
artillery  fires?"  or,  "What  is  the  'best'  utilization  of  Naval  fire 
support  in  amphibious  assaults?" 

In  a  previous  paper  [8] ,  we  have  pointed  out  that  the  structure 
of  the  optimal  allocation  policies  in  Lanchester  combat  is  basically 
determined  by  whether  there  are  constant  attrition  returns  over  time 
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per  unit  of  weapon  system  employed  or  diminishing  returns.   In  the 
present  paper  we  have  studied  target  selection  with  diminishing 
returns  over  time,  i.e.,  "linear-law"  attrition  process.   It  should 
be  noted  that  there  is  a  problem  in  the  literature  with  similar  solu- 
tion structures,  the  continuous  version  of  Bellman's  stochastic  gold- 
mining  process  (see  pp.  222-233  of  [1]).   When  there  are  diminishing 
returns  over  time  from  the  use  of  a  device  subject  to  breakdown,  then 
the  problem  of  maximizing  the  return  from  use  of  one  device  in  either 
of  two  potential  locations  has  a  similar  structure  to  the  optimization 
problem  in  Lanchester  combat  studied  here.   The  interested  reader 
should  compare  the  solution  as  shown  in  our  Figure  1  with  that  of 
Theorem  1  on  p.  231  of  [1]  and  also  our  Figure  2  with  Figure  4  on 
p.  232  of  [1].   When  the  stochastic  gold-mining  problem  is  re-examined 
by  modern  optimal  control  theory,  new  insights  are  gained  into  the 
operation  of  maximizing  the  return  from  a  resource  subject  to  break- 
down or  loss,  and  we  shall  discuss  this  in  the  future. 
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Appendix  E.   The  Theory  of  State  Variable  Inequality  Constraints. 

1.   Introduction. 

Optimal  control  problems  involving  inequality  constraints 
on  a  function  of  the  state  variables  with  no  explicit  dependence 
on  the  control  variables  have  been  treated  only  in  recent  years. 
The  pioneering  work  of  the  Russian  R.  Gamkrelidze  in  1959  (see 
Chapter  VI  in  [22])  was  followed  in  the  U.S.A.  by  that  of  L.  Berko- 
vitz  [3],  S.  Dreyfus  [12],  and  Bryson,  Denham,  and  Dreyfus  [9]. 
Berkovitz  and  Dreyfus  [5]  subsequently  have  shown  the  equivalence 
of  the  results  of  Gamkrelidze  and  Berkovitz  with  those  of  Dreyfus. 
Mclntyre  and  Paiewonsky  [20]  have  written  an  excellent  survey  ar- 
ticle on  the  theory  of  state  variable  inequality  constraints  (SVIC) 
and  summarize  theoretical  results  through  1966. 

Furthermore,  in  1967  Mclntyre  and  Paiewonsky  [20]  remarked 
that  "the  optimal  control  problem  with  state  space  constraints 
does  not  appear  to  be  well  understood."  Our  own  experience  bears 
this  out.   We  have  recently  pointed  out  [27]  that  certain  theoret- 
ical results  cited  in  the  literature  are  inadequate  by  themselves 
to  determine  optimal  trajectories  in  a  certain  non-autonomous 
class  of  problems  but  must  be  supplemented  by  a  result  due  to 
Gambrelidze  [22]. 

Even  more  incredible  is  the  fact  that  for  two-sided  varia- 
tional problems  (i.e.  two-person  zero-sum  deterministic  differen- 
tial games)  we  are  not  aware  of  any  adequate  treatment  of  inequality 
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constraints  of  functions  of  the  state  variables  only  (the  control 
variables  not  explicitly  appearing).   Although  R.  Isaacs  does  con- 
sider such  a  problem  (the  "War  of  Attrition  and  Attack"  [16]),  we 
will  show  that  his  solution  is  unsupported  by  proper  analysis,  and 
we  will  indicate  a  similar  problem  for  which  his  approach  fails  to 
yield  optimal  strategies.   L.  Berkovitz  does  not  consider  problems 
with  state  variable  inequality  constraints  (SVIC)  in  [4] ;  and  al- 
though A.  Friedman  claims  to  treat  such  problems  [13]  (see  also 
Chapter  6  in  [14]),  he  fails  to  develop  the  appropriate  necessary 
conditions  of  optimality  which  must  hold  when  such  a  constraint  is 
active  (i.e.  on  a  constrained  subarc) . 

In  this  appendix  we  shall  review  some  necessary  conditions 
of  optimality  which  must  hold  on  a  constrained  subarc  in  a  one- 
sided problem  with  a  SVIC.   Following  the  approach  of  L.  Berkovitz 
14],  we  may  readily  extend  these  conditions  to  differential  games. 
We  propose  this  to  ONR  as  a  future  research  task. 

2 .   Basic  Approaches. 

There  are  two  basic  approaches  to  treating  a  SVIC  in  an 
optimal  control  problem.   The  first  and  more  direct  approach  con- 
sists in  adjoining  the  state-variable  constraint  directly  to  the 
criterion  functional  with  an  additional  Lagrange  multiplier.   This 
approach  has  been  considered  by  Chang  [10]  and  Speyer  and  Bryson 
[23]  (see  also  [18]).   We  will  not  consider  this  method  of  direct 
adjoining  further  in  this  appendix. 
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The  second  method  consists  in  adjoining  the  time  derivative 
of  the  state-variable  constraint  to  the  criterion  functional  with 
an  additional  Lagrange  multiplier.   We  say  [9]  that  the  problem 
has  a  k    order  state  variable  inequality  constraint  (SVIC)  when 
the  first  time  (total)  derivative  of  the  state-variable  constraint 
which  explicitly  contains  the  control  variable(s)  is  the  k 
This  is  the  approach  originally  considered  by  Gamkrelidze  [22]  for 
a  first  order  SVIC  and  further  extended  to  higher  order  SVIC  by 
Bryson  et  al.  [9].   Furthermore,  this  is  the  approach  that  we  shall 
take  in  this  appendix. 

3.   Necessary  Conditions  of  Optimality  When  Time  Derivative  of 
Constraint  Is  Adjoined  to  Return  Functional. 

It  suffices  to  consider  a  problem  with  a  single  control 
variable  and  a  single  inequality  constraint  on  the  state  variables. 
Without  loss  of  generality  we  may  assume  that  the  problem's  plan- 
ning horizon  is  for  a  fixed  period  of  time.   We  further  consider 
a  (possibly  different)  inequality  constraint  on  the  state  variables 
at  the  end  of  the  planning  horizon. 

Let  us  therefore  consider  the  problem 


T 
maximize   A(x.(T))  +  /  L(t ,x. (t) ,u (t))dt ,  with  T  specified 
u(t)       X        0      X 
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dx 

subject  to:  -r—  =  f.(t,x.,u)     for  i  =  l,...,n,     (1) 


u(t)  e  U        (u(t)   is  scalar  control  variable 
suitably  restricted)  , 

C(t,x.)  ^  0     (scalar  inequality  constraint  on 
state  variables) , 

iJ;(x.(T))  £  0    (scalar  inequality  constraint  on 
state  variables  at  terminal 
time) , 


where  we  assume  that  all  functions  are  smooth  enough  to  insure  the 
existence  of  all  partial  derivatives  required  in  the  following 
analysis.   We  shall  consider  separately  the  cases  of  a  first  order 
SVIC  and  a  kth   (k  >  1)   order  SVIC. 

a.   First  Order  SVIC. 

Adjoining  the  first  time  derivative  of  the  state-variable 
constraint  to  the  return  functional,  we  have  that  the  Hamiltonian 
is  given  by  [8],  [9],  [11],  [22] 


n 


dC 


H(t,x.,Pl,u)  =  L(t,x.,u)  +  I      Pif.(t,x.,u)  -  u(t)  ~  ,   (2) 

i=l 


where 


u(t) 


=  0     for  C(t,x.)  <  0  , 


^  0     for  C(t,x.)  =  0  . 

l 
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We  may,  of  course,  write  (2)  in  expanded  form  as 


n 
H(t,xi,pi,u)  =  L(t,xi,u)  +  I      pif_L(t,xi,u) 

i=l 


3C    v   3C 


•>^{t  +  Z  gr  fi<»-*i.»»  ■  (3> 

i=l   1 


The  adjoint  equations  for  the  dual  variables  are  given  by 


dpi     BH      3L     r  ^±  ,,    3   /dC\       ,.. 

dT  =  "  ^7  =  "  ^xT  "  I     ^T  Pj  +  y(t)3xT  U)  '     (A) 
l       l   3  =  1   l  l 


where  p.(t)   is  the  dual  variable  corresponding  to  the  state  var- 
iable x..   Let  us  assume  (as  is  invariably)  that  all  state  variables 
are  specified  at   t  =  0,   i.e.  we  are  given  x.(t=0)  =  x.   for 
i  =  l,...,n.   Hence,   p.(t=0)   is  unspecified,  and  we  have  the  fol- 
lowing boundary  conditions  for  the  dual  variables  at   t  =  T 


l        x 


for   i  =  l,...,n  where 


=  0     for  tKx.(T))  <  0  , 


^  0     for  iJ;(x.(T))  =  0  . 


Conditions  (5)  were  apparently  first  observed  by  Funk  and  Gilbert 
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[15].   (A  special  case  of  conditions  (5)  was  earlier  observed  by 
Arrow  and  Kurz  [2]  although  their  derivation  is  only  heuristic.) 
Its  proof  readily  follows  along  the  lines  of  L.  Neustadt's  develop- 
ment in  {21],   (It  should  be  noted  that  (5)  has  been  stated  for 
the  prescribed  duration  problem.   It  need  not  hold  in  other  cases 
(for  example,  in  the  case  when  the  planning  horizon  ends  when  a 
target  set  (exclusive  of  time)  is  reached).) 

On  a  constrained  subarc  on  which  C(t,x.(t))  =  0   for 
O^t-^t^t-iST  we  have  that 


AC 

—  (t,x±(t))  =0     for  t±  <    t  <    t2  , 


and  hence  the  control  is  determined  by 


3C    ?   3C 


t  +  J,  t:  £i<c-vu  >  =  °  •  (6> 


3- 

i=l   i 


The  Lagrange  multiplier  p(t)  is  determined  by 

3H                 9L          r  3fi                     3     /dC\ 

*T  -  °  =  ^T  +     I  Pi  Iu~  "  ^(t)  ^  Idl)  *                  (7) 
i=l 

Bryson,  Denham,  and  Dreyfus  [9]  derived  that  on  a  constrained  sub- 
arc  (of  finite  length  in  time)  a  necessary  condition  of  optimality 
is 


y(t)  :>  0     for   t-  £  t  £  t2  .  (8) 
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For  this  problem  (1)  Gamkrelidze  (see  Chapter  VI  in  [22])  (who  did 
not  observe  (8))  developed  the  following  additional  necessary  con- 
dition 


;co-£*o 


for  t  £  t  £  t 


(9) 


We  have  recently  pointed  out  [27 J  that  many  workers  [1] ,  [8] ,  [9] , 
[11]  have  overlooked  the  importance  of  (9).   It  should  be  noted 
that  results  corresponding  to  (8)  and  (9)  have  not  been  developed 
for  differential  games  (see,  in  particular,  [4],  [14],  [16]). 

Furthermore,  there  are  corner  conditions  (for  a  discussion 
of  corner  conditions  within  the  framework  of  the  classical  calculus 
of  variations  see  p.  38  of  [6]  or  p.  367  and  p.  571  of  [7])  that 
must  be  satisfied  upon  entering  to  and  exiting  from  a  constrained 
subarc.   (These  important  conditions  are  never  mentioned  in  [1]  or 
[2].)  Let   t..   denote  the  entry  time  to  the  constrained  subarc  and 
t„   denote  the  exit  time.   This  situation  is  depicted  in  Figure  1. 


C=0 


t=T 


time,  t 


Figure  1.   Entry  to  and  Exit  from  Constrained  Subarc. 
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Furthermore,  let   t,    denote  a  left-hand  limit,  i.e.   t   =  lim  t 

**! 

^1 
It  has  been  established  [9]  that  the  total  discontinuity  in  the 

adjoint  variables  can  be  taken  at  the  entry  corner  (i.e.   t  =  t,). 

Thus ,  we  have  at  entrance  corner 


—         +        +   sC 
Pi(t1  )  =  pi(t1  )  -  uC^  )  —  (t1,xi)   for  i  =  l,...,n  ,  (10) 

i 


and 


H(tx  )  =  H(tl+)  +  y(tl+)  ~   (t1,x.)  .  (11) 


A  similar  result  was  first  given  by  Mclntyre  and  Paiewonsky  [20]. 
However,  there  is  a  sign  error  in  their  equation  (52).   Furthermore, 
the  dual  variables  can  be  taken  to  be  continuous  at  an  exit  corner. 
Thus,  we  have  that  at^  exit  corner 

Pi(t2")  =  P±(t2+)     for  i  =  l,...,n  ,  (12) 

and 


H(t2  )  =  H(t2+)  .  (13) 


It  is  readily  shown  that  (12)  and  (13)  imply  that 


y(t2  )  =  0  .  (14) 
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b.   Higher  Order  SVIC. 

In  the  case  of  a  k    (k>l)   order  SVIC,  the  first  time 
derivative  of  the  state-variable  constraint  which  explicitly  con- 
tains the  control  variable  is  the  k   .   Thus,  on  a  constrained 
subarc  on  which   C(t,x.(t))  =  0   for  0  £  t  £  t  £  t„  £  T   the 
control  is  determined  by 


^  =  0     for  t  <  t  <  t2  ,  (15) 

dt 


and  we  further  have 


for   tn  <  t  <  t-  , 

^=  0 

dtn 

for  n  =  l,...,k  -  1  .  (16) 

The  Hamiltonian  is  then  given  by  [8] ,  [9] 


n  dkc 

H(t,x  ,p  ,u)  =  L(t,x  ,u)  +  I      p  f  (t,x  ,u)  -  y(t)  — r  ,   (17) 

1=1  dt 


where 


y(t) 


=  0     for  C(t,x.)  <  0  , 


;>  0     for   C(t,x.)  =  0  , 


and  the  adjoint  equations  for  the  dual  variables  are  given  by 


dpi     3H      3L 


dt      Sx.      9x. 

l       l   j=l    l 


1=1   i  J  i  *dt  ' 
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for   i  =  l,...,n.   Again,  we  assume  that  initial  conditions  are 
given  for  all  the  state  variables  (i.e.   x.(t=0)  =  x.   for 
i  =  l,...,n).   Then  the  boundary  conditions  at   t  =  T   for  the 
dual  variables  are  again  given  by  (5) . 

On  a  constrained  subarc,  the  control  is  determined  by  (15), 
As  derived  by  Bryson,  Denham,  and  Dreyfus  19],  a  necessary  condi- 
tion of  optimality  on  a  constrained  subarc  is 


y(t)  :>  0     for  t1  <;  t  <;  x.      ,  (19) 


where  the  Lagrange  multiplier   (t)   is  determined  by 


9H       9L    v  3fi    f    .   9  /dkC  \  ,om 

Iu"  "  °  =  ^  +    .?  t  ±  J^T  ~   M(t)^-  [~^     •  (20) 

i=l  %dt  ' 


It  remains  to  discuss  the  corner  conditions  (see  [9],  [20]). 
At  an  entrance  corner  we  have  that 

P.(tl   )    -  p    (t     )   +  vQ  —   (t  )   +     I     *     —l—\ 

i  n=l  i  xdt   ' 

and 

H(tl-)    -  H(tl+)    -  vQ  |f   (tl)    -  T   vn  £  g) 

+  -<h+>  &(£er)  ■  (22) 


where    .   for  j  =  0,1,..., k-2   are  undetermined  constants 
(multipliers) .   (Again,  our  results  (21)  and  (22)  differ  in  sign 
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with  the  corresponding  results  given  by  Mclntyre  and  Paiewonsky 
[20].)   Finally,  at  an  exit  corner  ve  simply  have 

Pi(t2")  =  Pi(t2+)     for   i  =  l,...,n  ,  (23) 


and 


H(t2  )  =  H(t2+)   ,  (24) 


which  imply  that 


y(t2  )  =  0  .  (25) 


4.   Application  of  Theory  to  Tactical  Allocation  Problems. 

In  this  section  we  apply  the  theory  discussed  above  to  two 
tactical  allocation  problems.   Both  these  problems  have  first  order 
state  variable  inequality  constraints.   The  first  problem  (tactical 
air-war  campaign)  is  a  one-sided  version  of  R.  Isaacs 's  "War  of 
Attrition  and  Attack  [16]."  However,  we  shall  consider  versions 
for  which  previous  analysis  [16]  does  not  lead  to  an  optimal  policy. 
The  second  problem  is  the  fire  programming  problem  studied  by 
Isbell  and  Marlow  [17].   (See  also  [25],  in  this  report  as  Appendix 
A,  where  results  are  obtained  in  a  heuristic  fashion  (the  determi- 
nation of  boundary  conditions  for  the  dual  variables  following  [17]) 
without  the  appropriate  theory  of  SVIC.)   We  shall  consider  several 
versions  of  both  these  problems.   We  will  not  develop  complete  solu- 
tions (including  synthesis  of  optimal  policies  as  a  function  of 
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initial  conditions)  to  these  problems  but  concentrate  on  developing 
the  basic  necessary  conditions  of  optimality. 

a.   Tactical  Air-War  Campaign. 

We  consider  a  one-sided  version  of  R.  Isaacs' s  "War  of  Attri- 
tion and  Attack  [16]."  We  consider  the  case  when  one  of  the  com- 
batant forces  always  flys  ground  support  missions. 

(1)   Original  Version  of  Isaacs. 
We  consider  the  problem  facing  the  X-force  commander 


T 
maximize  /   {x(l-<j))  -  y}  dt   with  T  specified, 
<Kt)    0 


,  .  dx 

subiect  to:  -r—   =  m,  , 
J         dt    1 


dt  =  m2  "  *C2X  ' 


x,y  ;>  0    and   0  :>  <j>  :>  1  ,  (26) 


where 


x  and  y  are  the  number  of  X  and  Y  aircraft,  respectively, 

c   is  the  rate  at  which  one  X  aircraft  shoots  down  Y  air- 
craft (Lanchester  attrition-rate  coefficient) , 


m 


,   and  m   are  replacement  rates, 

<j)   is  the  fraction  of  total  X  aircraft  which  fly  counter-air 
missions  (and  hence  shoot  down  aircraft) . 


E-13 


Clearly,  the  constraint  x  ^  0   can  never  be  binding.   The  Hamil- 
tonian  is  given  by 

H(t,x  ,p  ,<|>)  =  x(l-cf))  -  y  +  p^  +  p2(m2-<j)C2x) 

+  y(t)(m2-<|)C2x)  (27) 

where 


and  p..   is  the  dual  variable  corresponding  to  x.   The  adjoint 
system  of  differential  equations  for  the  dual  variables  is 


dpx 


—L  =  -(1-$)  +  <|>c2(p2+m)  ,  (28) 

dp9 

"  =  1  ,  (29) 


dt 

with  boundary  conditions  at   t  =  T 

Pl(t=T)  =  0  ,  (30) 

since  x(T)  >  0   (assuming  T  >  0)   and 


P2(t=T)  =  v2  ,  (31) 


where 

for  y(T)  >  0  , 


V2 


(^  0     for 


y(T)  -  0  . 


E-14 


When  y  >  0,   the  control  law  is  determined  by  the  maximum 
principle.   This  leads  to 

for   (-p  (t))  >  ± 
2 

for   (-p9(t))  <  —  .  (32) 

z       c2 


The  possibility  of  a  singular  solution  (see  Appendix  D  for  a  further 

dp2 
discussion  is  excluded  by  the  fact  that  since   ,   ^  0,   it  is  im- 


possible  to  have  -7—  It—)  =  0  when  —  =  0.   Hence,  the  extremal 

at\d<p  /  9  9 

control  (it  is  also  optimal,  since  we  shall  see  that  there  is  only 
one  extremal)  is  well-defined  by  the  control  law  (32)  almost  every- 
where in  time  (i.e.  the  extremal  control  is  determined  by  (32) 
except  at  a  finite  number  of  isolated  instants  in  time  when 

(-P2(t))  =±   ). 
C2 

Furthermore,  the  locus  of  points  for  which   (-p9(t))  =  — 

c 

a  * 

(and  $  (t)   consequently  changes  from  1   to  0  as  an  extremal 

"penetrates"  this  surface)  may  be  referred  to  as  a  "switching"  or 

"transition"  surface.   The  discontinuity  in  the  control  at  such  a 

locus  implies  that  the  trajectory  in  the  state  space  has  a  corner 

(without  any  path  constraint) ,  i.e.  a  discontinuous  change  in  the 

slope  of  the  state  variable  trajectory.   There  is  no  tangent  to 

the  path  at  the  switching  time,  although  both  the  left-hand  and 

right-hand  derivatives  do  exist.   It  is  well-known  (see,  for  example, 

p.  125  of  [8]  and  also  references  to  the  classical  calculus  of 
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variations  in  Section  3.  above)  that  the  following  conditions  must 
hold  at  such  a  corner 


P±(t  )  =  p±(t  )    for  i  =  l,...,n  , 


H(t  )   =  H(t+)  , 


where   t   is  the  time  just  before  the  corner  (left-hand  limit). 
It  should  be  noted  that  these  vital  corner  conditions  are  never 
mentioned  in  [1],  [2],  [16],  or  [24].   However,  in  all  these  refer- 
ences they  are  always  implicitly  assumed.   As  we  will  presently 
see  below,  the  corner  conditions  may  take  a  different  form  at  the 
entrance  to  a  constrained  subarc  (i.e.  problem  with  SVIC  when 
Gamkrelidze's  approach  (see  pp.  265-267  of  [22])  is  used),  and  this 
implicit  assumption  is  then  no  longer  valid. 

On  a  constrained  subarc  on  which  y(t)  =  0   for  t  ^  t  £  t„ 

the  control  is  determined  by  -r^-  =  0  and  hence 

at 


*  m2 

*  (t)  =  c^TcEY        for    h  <  *  <  C2  '  (33) 


3H 
The  multiplier  u(t)   is  determined  by  the  condition  —  ■  0  and 

dtp 

hence 

u(t)  =  -  —  -  p  (t)  .  (34) 

c2 

The  condition  that   u(t)  ^  0   yields  that  on  a  constrained  subarc 

we  must  have 


E-16 


(-p  (t))  ^  -^  .  (35) 

2 

Differentiating  (34)  and  combining  with  (29) ,  we  find  that 

dP2 
u(t)  -  -  -j—-  -1  <  0  ,  (36) 

so  that  Gamkrelidze's  condition  is  always  satisfied  on  a  constrained 
sub arc. 

Denoting  the  time  of  an  entrance  corner  by  t  and  that  of 
an  exit  corner  by  t„  ,  the  corner  conditions  (10),  (11),  (12),  and 
(13)  yield  that 

P2(tl-)  =  -  ^  ,  (37) 


?2(ti+>  "  -  T2  -  *(ti+)  ■  <38) 

and 


p2(t2-)  =  p2(t2+)  =  -  i  (39) 


When  there  is  an  exit  from  a  constrained  subarc  at   t  =  t  ,   then 
(using  (14)  and  (36))  it  is  readily  shown  that  (38)  becomes 

?2(tl+)  "  "  7~2   -    (t2-Cl>  •  (40) 

Having  developed  the  basic  necessary  conditions  of  optimality, 
let  us  now  consider  the  synthesis  of  the  optimal  control.   By  the 
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synthesis  of  optimal  control,  we  mean  the  explicit  determination 
of  the  time  history  of  the  optimal  control  from  initial  to  terminal 
time  as  a  function  of  the  initial  state  of  the  system.   In  synthe- 
sizing an  optimal  trajectory  there  are  two  cases  to  be  considered: 

Case  (a)     y(T)  =0  , 

Case  (b)     y(T)  >  0  . 

We  now  show  that  an  optimal  trajectory  cannot  end  at   t  =  T  with 
y(T)  =  0. 

For  Case  (a):   y(T)  =  0,   there  are  two  further  subcases, 
corresponding  to  whether  or  not  the  trajectory  was  on  a  constrained 
subarc  for  a  finite  period  of  time  immediately  before  the  end  at 
t  =  T.   Thus,  we  consider 


Subcase 


(i)     y(T)  =  0  with  y(t)  =0  for  t  <:  t  £ 


where   t.  <  T. 


In  this  case 


p9(t=T)  =  -  —  -  y(t=T)  ,  (41) 

2 


where 


y(t=T)  ^  0  , 

since  we  are  on  a  constrained  subarc.   However,  it  is  readily  shown 
that  this  is  incompatible  with  (31)  which  may  be  re-stated  as 
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P2(t=T)  =  v2  (42) 

where 

v2  :>  0  . 

Hence,  Subcase  (i)  is  impossible  by  the  inconsistency  of  (Al)  and 
(A2)  ,  both  of  which  are  necessary  conditions  of  optimality. 

Subcase  (ii)    y(T)  =  0  with  y(t)  >  0   for  T  -  6  £  t  <  T 
where  6  >  0. 

Since  y(T)  =  0  with  y(t)  >  0   for  T  -  6  <;  t  <  T  where 
6  >  0,  we  must  have  (considering  the  state  equations  (26)) 

4»  (t)  =  1     for  T  -  6  <;  t  <  T  .  (A3) 

By  (31)  (or  (42))  and  (29),  we  have 


where 


P2(t)  =  v2  -  (T-t)  (AA) 


v2*  0  . 


Thus,  we  can  find   e  >  0   such  that  for  T  -  e  ^  t  <  T,  we  have 

P2(t)  >  -1  ,  (45) 

2 

and  hence  considering  the  control  law  (32)  ,  we  are  led  to  a  contra- 
diction of  (A3).   Thus,  Subcase  (ii)  is  similarly  impossible.   Hence, 
optimal  trajectories  are  only  possible  in  Case  (b) .   It  had  not  been 
shown  previously  [16]  that  Case  (a)  is  not  consistent  with  an  optimal 
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policy.   (In  fact,  the  possibility  of  Case  (a)  is  not  even  considered 
in  [163,  [24].) 

For  Case  (b) :   y(T)  >  0,   we  have  by  (31)  that 

P2(t=T)  =  0  .  (46) 

Using  this  terminal  condition,  a  backwards  integration  of  the  adjoint 
equation  (29)  yields 

(-p2(x))  =  t  ,  (47) 

where,  for  convenience,  we  have  introduced  the  backwards  time  t 
defined  by  x  =  T-t.  Hence,  in  the  case  when  y(t)  >  0  for  all 
t,   it  is  easily  seen  that 

for  x  >  —  , 
C2 

for  t  <  -f-  ,  (48) 

2 

where  we  have  considered  (32)  and  (47).   Thus,  in  this  special  case 

(when  y(t)  >  0   for  O^t^T  )   the  solution  shown  in  Table  I 

C2 
readily  follows.   Combining  the  solution  shown  in  Table  I  with  a 

forward  integration  of  the  state  equations  (26)  ,  it  is  easy  to  show 

that  this  solution  applies  under  the  following  conditions: 

either   (a)   m1c„(T )  <  m„  -  c„x.  ,  (49) 

1  z     c„      2.  I   U 

mlC2^T~  c~^  *  m2  ~  °2X0  ' 

C?mi         1  1 

VT<,-^t<'A^?'      (50> 


Table  I.   Optimal  Policy  in 

Tactical  Air-War  Campaign 
When  y(t)  >  0   for  all   t. 


Case  (1) :   T  £  — 
C2 


* 

<J>  (t)  =  0     for  0  <;  t  <;  T 


(Always  fly  ground  support.) 


Case  (2) :  T  >  — 
C2 


<}> 


(l     for  0  £  t 


for  T  -  —  £  t  <.   T 
C2 
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It  remains  to  develop  the  solution  in  Case  (b)  when  neither 
(49)  nor  (50)  holds.   Thus  when 


cm 
^0  *  —  (T"  ^   +  <c2X(Tm2)(T-  i>    •         (51) 


we  have  y(tn)  =  0  where   t,  <  T  .   In  this  case  y(t)  >  0 

1  1       c„ 

_i 
for  0  <;  t  <  t   and  recalling  (37)  that  p  (t  )  = ,   it  is 

*  2 

readily  seen  that   $  (t)  =  1   for  0  ^  t  <  t, .   Furthermore, 


*       m2  1 

<j>  (t)  =  — —     for   t,  sS  t  a!  T  -  -=-  .  (52) 

c2  x  1  c2 


We  observe  that  by  (40) 

p2(ti+>  -  -  r2  -  <T  -  r2  -  h>  •  <53) 

Clearly  (35)  is  satisfied.  Thus,  both  (8)  and  (9)  are  satisfied  so 
that  it  is  optimal  "in  the  small"  to  remain  on  the  constrained  sub- 
arc.  Finally,  it  is  readily  shown  that  (39)  and  (46)  are  consistent 
The  final  solution  (the  extremal  trajectory  is  optimal,  since  it  is 
unique)  may  be  expressed  in  a  particularly  simple  form  and  is  shown 
in  Table  II. 

(2)   A  Time -Dependent  Attrition-Rate  Coefficient. 

If  the  approach  used  by  Isaacs  [16]  is  applied  to  the  above 

problem,  then  it  is  assumed  that  it  is  optimal  to  drive  y(t)   to 

zero  and  keep  it  there  for   t  £  T  whenever  possible.   However, 

c2 


Table  II.   Optimal  Policy  in 

Tactical  Air-War   Campaign 


Case  (1)  :  T  <:  — 
C2 


>  (t)  =  0     for  O^t^T 


(Always  fly  ground  support.) 


Case  (2):  T  >  — 
C2 


For  0  £  t  :£  T 

C2 


when  y  >  0  , 


1  * 

For  T £  t  £  T  ,     cf>(t)=0. 

C2 
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this  need  not  be  true  and,  indeed,  is  not  true  in  general  as  the 
following  example  shows.   We  consider  the  case  when  the  attrition- 
rate  coefficient  c„   is  a  function  of  time. 


T 
maximize  J   (x(l-<)0  -  y)  dt     with  T   specified  , 
4>(t)    0 


.  .         dx 
subject  to:   -r-  =  mn  , 
J         dt    1 


*  =  m2  -  <|>  c2(t)  x  , 


x,y  ^  0     and     8  £  $  £  i  .  (54) 

We  will  consider  only  the  development  of  the  basic  necessary 
conditions  of  optimality.   The  analysis  of  the  preceeding  section 
applied  through  (35).   However,   c~   is  now  a  function  of  time. 
Re-writing  (34)  as 

*(t)  =  -^k-f2(t)  •  (55) 

we  have 

1   dc2 

*(t)  =  (c^dt1-1  ■  (56) 

so  that  Gamkrelidze's  condition  (9)  is  only  satisfied  on  a  con- 
strained sub arc  when 


dc 


2  .  r       ,..„2 


£  [c0(t)]2   .  (57) 


dt  "  L  2 
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The  corner  conditions  (10)  through  (13)  now  take  the  form 


p2(tr)--^y>  (58) 

"2(ti+)  -  -  ^Ti^  -  ^(ti+)   •  (59) 

and 


p2(t2")  =  p2(t2+)  =  -  ^-y  .  (60) 


Considering  (14)  and  (56)  ,  it  is  easily  seen  that 


and  hence  for  t,  ^  t  ^  t_ 


n(t)  =  t9  -  t  +  )     .    -  — ~y  •  (61) 

2       c2      T  2 


Considering  (61)  ,  we  see  that  when  there  is  an  exit  from  a  con- 


strained subarc  at   t  =  t?,  we  have 


»2(tl+>  "  "  ^T  -  (t2-<l>  •  <«> 
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(3)   Linearly  Increasing  Attrition-Rate  Coefficient. 
It  seems  appropriate  to  consider  a  special  case  of  a  time- 
dependent  attrition-rate  coefficient  in  order  to  more  clearly  illus- 
trate that  there  are  restrictions  on  when  it  is  optimal  to  keep 
y(t)   equal  to  zero  for  a  finite  interval  of  time.   Let  us  therefore 
consider  the  special  case  when  c  (t)   is  a  linear  function  of  time, 
i.e. 

c2(t)  =  kt  .  (63) 

In  this  case,  (57)  holds  (this  implies  that  Gamkrelidze's  necessary 
condition  (9)  is  satisfied  on  a  constrained  subarc)  only  for 


t  ^  —  .  (64) 


Furthermore,  (58)  now  becomes 


»2<*r>  -  -  ^  •  (65> 


so  that  by  using  (29)  we  find  that  for  0  <:  t  <.   t. 


P2<C>  =-^+t"  h  •  (66) 


Considering  the  control  law  (32) ,  the  above  expression  (66)  yields 
that 
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for  0  £  t  < 


I  ktl  ' 

1     for  j~  <  t  <  t- 


Let  us  denote  the  time  of  switch  in  tactics  by   t   so  that  we  have 

s 


Cs=k^  •  (68) 


Integration  of  the  state  equations  (54)  using  (67)  then  yields 


for   t  <.   t  ^  t.. 
s        1 


y(t)  =  y0  +  V  -  -2-  (t*-l*)  -  -i-  (t»-tj)  .       (69) 


The  unknown  quantity  t   can  then  be  determined  by  solving  th< 
equation 


y(tx)  =  0  ,  (70) 


where  y(t)   is  given  by  (69)  and   t   is  given  by  (68) .   We  have 

not  worked  out  further  details. 

Furthermore,  if  we  are  never  on  a  constrained  subarc  then 

1 
11     tor  T-t  >  - 

f"(t)  = 


for  T-t  <  ~-     .  (71) 

kt 
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We  observe   that 


and 


4>    (t=0)    =  0    , 


$    (t«T)    =  0    . 


If  there  is  a  switch  in  tactics,  then  (ignoring  the  pathological 

case)  there  must  be  two,  which  we  denote  as   t    and  t   .   These 

Sl        S2 
are  the  roots  of  the  equation 


T  "  cs  -  kT  •  (72) 

S 


2 
Consequently,  for  T  £  —   we  have 


<J>*(t)  =0     for  O^t^T.  (73) 


2 
Also,  for  T  >  —  we  have 

/k 


where 


and 


1° 

vo 

for     0  <;  t  < 

for     t       <   t 

sl 
for      t        <    t 

s2 

V 

<  t 

S2 
£  T    , 

T 
Sl        2 

-  V(f)2  - 1 

> 

T 
s2        2 

*  V(ir  -  * 

• 

(74) 


(75) 


(76) 
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It  seems  appropriate  to  briefly  discuss  what  this  partial 
analysis  has  yielded.   First  of  all,  we  (the  X-forces)  don't  begin 
the  campaign  by  shooting  down  Y  aircraft  but  wait  until  our  ef- 
fectiveness rises  enough  over  time.   Intuitively  we  may  think  of 
this  as  manifesting  that  we  should  not  shoot  down  planes  today 
(initially)  because  we  can  do  so  much  better  tomorrow.   Secondly, 
we  adjust  our  tactics  so  that  (64)  holds  when  we  keep  the  number 
of  Y  aircraft  at  the  zero  level.   We  note  that  we  don't  always 
drive  y(t)   to  zero  whenever  we  can,  but  we  wait  so  that  (64)  can 
hold.   We  have  not  worked  out  the  conditions  on  the  initial  force 
levels  under  which  a  constrained  subarc  occurs. 

(4)   Time  Variations  in  Integrand  of  Payoff. 
Let  us  further  consider  an  example  in  which  the  payoff  from 
allocating  aircraft  to  ground  support  missions  can  change  over  time, 
In  other  words,  we  try  to  take  account  of  changing  effectiveness  of 
the  supporting  weapon  system  (here  aircraft) .   The  model  is 


T 
maximize  /   {a(t)x(l-<}))  -  b(t)y}  dt   with  T   specified  , 


<J>(t)   0 


,  .         dx 
subject  to:   —  =  m   , 


dt  =  m2  "  *C2X  ' 


x,y  ^  0     and     0  <;  <f>  ^  1  ,  (77) 

where   c„   is  again  assumed  to  be  constant. 
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Again,  we  shall  focus  on  the  development  of  basic  necessary 
conditions  of  optimality  and  an  interpretation  of  the  parameters 
on  which  the  optimal  policy  depends.   The  Hamiltonian  is  given  by 

H(t,xi,pi,(())  =  a(t)x(l-<p)  -  b(t)y  +  p  m  +  p2(m2-ct>c2x) 

+  y(t)(m2-<K2x)  ,  (78) 

where 


and  the  adjoint  equations  for  the  dual  variables  are 

dpl 

^  =  -  a(t)(l-cp)  +  <pc2(p2+y)  ,  (79) 


dp2 
dT~ 


=  b(t)  ,  (80) 


with  boundary  conditions  at   t  =  T  again  given  by  (30)  and  (31) . 
When  y  >   0 ,   the  control  law  is  determined  by  the  maximum 
principle.   This  leads  to 

1     for   <-p  (t))  >  4^  , 

2 


♦  (t)  = 


0     for  (-P?(t))  <  2&1  .  (81) 

c2 
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The  probability  of  a  singular  solution,  however,  is  not  as  easily 
excluded  as  before.   Although  it  is  readily  shown  to  be  possible 


to  have  —  (— 1  =  0  when  — -  =  0,  complete  details  as  to  whether 

dt\3(j)/  dq> 


there  can  be  a  singular  subarc  haven't  been  worked  out  at  this 
time. 

Again,  on  a  constrained  subarc  on  which  y(t)  =  0   for 
t,  £  t  •£  t.   the  control  to  keep  y(t)  =0   is  given  by  (33)  .   The 
multiplier  y(t)   is  determined  by  the  condition  —  =  0   and 

dcp 

hence 

y(t)  =  -'^i£l-  p  (t)  .  (82) 

c2 

The  condition  that  u(t)  ^  0  yields  that  on  a  constrained  subarc 
we  must  have 


(-p„(t))  ^  ^P-  (83) 

2 


Differentiating  (82)  and  combining  with  (80),  we  find  that 


U(t)  =  -  —  ^  -   b(t)  ,  (84) 

c_  dt 


so  that  Gamkrelidze's  condition  (9)  is  only  satisfied  on  a  con- 
strained subarc  when 


b(t)  *  --r-7?-  •  (85) 

c?  dt 
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The  corner  conditions  (10)  through  (13)  now  yield 


a(tx) 


and 


P2(tl  )  =  "  ~ '         '         (86) 


+      a(tl)       + 
P2(tl  }  =  "  ~c" y(tl  )  »  (87) 


+              a(t2} 
P2(t2   )    =  p2(t2   )    =  -  — .  (88) 


Considering  (14)  and  (84) ,  it  is  easily  seen  that  for  t-  £  t'  «S  t- 


1  l? 

u(t)  =  —■  {a(t  )  -  a(t)}  +  /   b(s)  ds  ,         (89) 

C2  t 


and  thus  when  there  is  an  exit  from  a  constrained  sub arc  at   t  =  t„  , 
we  have 


a(t?)    t 

p  (t  )  = —  -  j       b(s)  ds  .  (90) 

°2    t 

From  the  above  analysis  we  see  that  the  "weight"  of  the 
ground  support  missions,  i.e.   a(t)   and  b(t),   are  determining 
factors  in  ascertaining  optimal  allocation  tactics.   From  this  we 
conclude  that  another  type  of  model  (one  that  considers  the  conse- 
quence of  allocating  aircraft  to  ground  support  missions)  may  be 
more  appropriate.   In  particular,  the  model  (26)  (due  to  A.  Mengel) 
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ignores  the  dynamics  of  ground  combat  (i.e.  the  interaction  of  the 
supporting  X   and  Y  weapon  systems  with  the  battle  of  ground 
forces) .   We  propose  this  to  ONR  as  a  future  research  task. 

b .   The  Isbell-Marlow  Fire  Programming  Problem. 

We  consider  a  problem  about  the  optimal  distribution  of  fire 
over  enemy  target  types.   This  problem  was  first  studied  by  Isbell 
and  Marlow  [17J  (the  same  analysis  is  apparently  given  in  [24]). 
However,  we  have  shown  that  the  solution  given  in  [17]  is  not  valid 
for  all  ranges  of  model  parameters  [25]  (reproduced  in  this  report 
as  Appendix  A) .   (There  are  some  gaps  in  our  solution  development 
that  we  rectify  in  Appendix  F,  although  our  previous  solution  [25] 
is  not  modified.)   Additionally,  we  developed  in  [25]  a  purely 
algebraic  method  for  constructing  a  solution  to  such  problems,  as 
opposed  to  the  geometric  method  (which  failed  to  uncover  a  completely 
correct  solution)  employed  in  [17J. 

However,  in  [25]  we  still  used  the  same  heuristic  method 
introduced  by  Isbell  and  Marlow  [17]  to  determine  boundary  condi- 
tions for  the  dual  variables.   There  are  some  other  technical  gaps 
having  to  do  with  the  treatment  of  the  non-negativity  of  the  force 
levels  (state  variable  inequality  constraints) .   We  re-consider 
the  development  of  certain  key  necessary  conditions  of  optimality 
within  the  framework  of  the  theory  of  SVIC .   This  is  the  first 
such  application  of  the  theory  of  SVIC  to  such  problems  to  the 
best  of  our  knowledge. 
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(1)   Original  Version. 
We  consider  the  problem 

maximize  {ry(T)  -  px..  (T)  -  qx_(T)}  with  T  unspecified  , 
*(t)  L  2 

dxl 
subject  to:  -^-   =  -  ^y  , 

dx„ 

di Ci-*)a2y  , 

dt     =  -  blXl  -  b2X2  ' 
x^x-.y  ^  0    and    0  ^  <j>  £  1  , 

where 

x-Ct),  x2(t),  y(t)   are  force  levels, 

p,  q,  r  are  utilities  assigned  survivors, 

a  ,  a_ ,  b  ,  b^  are  (constant)  attrition-rate  coefficients, 
and    <j>   is  the  fraction  of  Y  fire  directed  at  X,  . 

The  battle  terminates  at   t  =  T  with  either  (a)   x  (T)  =  x  (T)  =  0 
or  (b)   y(T)  =  0   (i.e.  it  is  a  "fight  to  the  finish").   As  in  [25] 
we  shall  denote  the  five  terminal  states  of  battle  corresponding  to 
the  above  two  conditions  as  C.   for   i  =  1,...,5   (see  [25]  for 
precise  definitions) . 

We  will  focus  on  the  development  of  necessary  conditions  of 
optimality.   The  synthesis  of  optimal  trajectories  from  these  is 
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carried  out  in  Appendix  A  (see  also  Appendix  F) .   The  Hamiltonian 
is  given  by 

H(t,xi,pi,<j))  =  -p1<j>a1y  -  p2(l-<}>)a2y  -  p3(b1x1+b2x2) 

-  y1(t)<|)a1y  -  y2(t)(l-<|>)a2y  ,  (92) 


where 


U±(t) 


=  0     for  x.  >  0  , 
l 


k  0     for  x.  =  0 

x 


The  adjoint  system  of  differential  equations  for  the  dual  variables 


xs 


dpi 

IT  "  Vs  •  (93) 

dp 

IT  -  Va  •  (94> 

dp 

—  =  <|)a1(p1+y1)    +    (l-4>)a2(p2+y2)    .  (95) 

Deferring  the  discussion  of  boundary  conditions  for  the  dual  var- 
iables until  later,  we  note  the  transversality  condition 

H(t=T,x.,p*,<J>*)  =  0  .  (96) 

When  x..  ,x_  >  0,   the  control  law  is  determined  by  the  max- 
imum principle.   This  leads  to 
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II     for  a^-p^t))  >  a2(-p2(t))  , 
0     for  a^-p^t))  <  a2(-p2(t))  .      (97) 

As  we  showed  in  [25] ,  it  is  easily  seen  that  it  is  impossible  to 

d/3H\    _   ,      3H  M      .      . 

have  TtItt)  =  0  when   —  =0   so  that  it  is  impossible  to  have 

a  singular  solution. 

Without  loss  of  generality,  we  may  consider  a  constrained 

subarc  on  which  x,(t)  =  0   (and  x?  >  0)   for   t-  £  t  £  t„.   The 

control  is  clearly  $  (t)  =  0.   The  multiplier  M.(t)   is  determined 

by  the  condition  —  =  0   and  hence 

U1(t)  =  —  (a2p2-a1p1)  •  (98) 

The  condition  that  y,(t)  ^  0  yeilds  that  on  a  constrained  subarc 
we  must  have 


a.C-p^t))  :>  a2(-p2(t))  .  (99) 


Differentiating  (98)  and  combining  with  (93)  and  (94) ,  we  find  that 


P3<t) 

y(t)  =  -^ (a2b2-a1b1)  ,  (100) 


so  that  Gamkrelidze's  condition  (9)  is  only  satisfied  on  a  constrained 
subarc  with  x,  =  0  when 

albl  *  a2b2  '  (101) 
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since  it  is  readily  shown  that   Po(0  ^  0«   The  corner  conditions 
(10)  and  (11)  at  an  entrance  corner  yield  that 

Pl(tl_)  =  Pl(tl+)  +  yl(tl+)  '  (102) 

P2(tl_)  =  p2(tl+)  '  (103) 

p3(tl")  =  P3(tx+)  ,  (104) 

and 

Pl(tP  '  \   P2(tl")  •  (105) 

Let  us  now  consider  the  boundary  conditions  on  the  dual  var- 
iables at   t  =  T.   We  use  the  same  notation  for  the  terminal  states 
of  battle  as  used  in  {25]  (reproduced  in  this  report  as  Appendix  A). 
Without  loss  of  generality,  we  can  assume  that   a  b_  >  a  b  .   Th e n 
Gamkrelidze's  condition  (101)  implies  that  no  extremals  can  lead  to 
C„,  and  it  may  be  shown  that  there  is  no  inconsistency  with  having 
extremals  lead  to  C  .   Terminal  states   C  ,  C  ,  and  C,.   are  treated 
in  a  common  fashion,  since  for  all  of  them  the  time  of  the  end  of 
battles  is  determined  by  y(T)  =  0.   Hence,   y(T)  =0   is  a  specified 
equality  condition  on  a  state  variable  at  an  unspecified  terminal 
time.   Considering  this  fact  (see  p.  75  of  [8])  and  (5),  we  have 
for  states   C  ,  C  ,  and  C,. 
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where 


Pl(t=T)  =  -p  +  vi  (106) 


=  0     for  x  >  0  , 


v 
1 


where 


v 
2 


>   0     for  x  =  0  , 


P2(t=T)  =  -q  +  v2  (107) 


=  0     for  x„  >  0  , 


^  0     for  x  =  0  , 


P3(t=T)  =  v3  (108) 

where   v„   is  unrestricted. 

Furthermore,  the  transversality  condition  (96)  yields  in  all  these 
cases  (i.e.  for  C  ,  C,,  and  C,.) 

P3(t=T)  =  0  .  (109) 

In  particular  for   C, :   x  (T)  >  0  ,  x  (T)  =  0,  y(T)  =0,   it 
is  readily  seen  by  (101)  that  this  terminal  state  cannot  be  reached 
by  being  on  a  constrained  subarc  with  x_  =  0   for  a  finite  interval 
of  time.   Thus,   C,   can  only  be  reached  when  x_(T)  =  0  but 
x»(t)  >  0   for   t  <  T.   The  boundary  conditions  (106)  and  (107)  on 
the  dual  variables  now  take  the  form 
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Pl(t=T)  =  -p  ,  (110) 

and 

P2(t=T)  =  -q  +  v2  ,  (111) 

where   v  ^  0  . 

The  latter  condition  (111)  was  conjectured  to  hold  in  [25] ,   but  we 
could  not  at  that  time  justify  it.   Since  x„(T)  =  0  but  x~(t)  >  0 
for  t  <  T,  we  must  have  tf)  (T)  =  0,   and  hence  by  (97)  we  have 


alP 
(-p(t=T))  >  -~-  .  (112) 

a2 


Then,  combination  of  (111)  and  (112)  yields  that 


<  (-p0(t=T))  £  q  ,  (113) 


axp 

which  was  a  result  used  in  [25]  that  we  could  not  justify  (due  to 
our,  at  that  time,  lack  of  knowledge  of  the  theory  of  SVIC) . 

(2)   A  Case  of  Time-Dependent  Attrition-Rate  Coefficients. 
As  we  discussed  in  [27]  ,  for  problems  with  autonomous  dynamics 
(8)  may  well  imply  that  Gamkrelidze's  condition  (9)  is  satisfied.   In 
non-autonomous  cases  this  need  not  be  the  case  and  highlights  the 
importance  of  Gamkrelidze's  condition.   This  work  here  complements 
our  results  presented  in  [26]  (reproduced  in  this  report  as  Appendix  C) , 
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where  we  omitted  considerations  pertaining  to  the  non-negativity  of 
the  force  levels.   Let  us  therefore  consider  the  problem 


maximize  (ry(T)  -  px  (T)  -  qx  (T)}   with  T   specified  , 

<Kt) 

dxi 

subject  to:  ^-  =   -(fra^Oy  +  r1(t)  , 


dx„ 

^—  =   -(l-<j))a2(t)y  +  r2(t)  , 


-gL     =   -b1(t)x1  -  b1(t)x2  +  s(t)  , 


x.,x2,y  ^  0     and     0  <:  <f>  ^  1    ,  (114) 


where  r  (t) ,  r„(t)  ,  s(t)   are  replacement  rates.   Again,  we  focus 
on  the  development  of  some  necessary  conditions  of  optimality. 
The  Hamiltonian  is  given  by 


H(t,xi,pi,(j))  =  p1(t){-<j)a1(t)  +  r1(t)}  + 
p2(t){-(l-<j>)a2(t)y  +  r2(t)}  +  p3(t)  { -b  ±(  t)x±   -  h^t)^   +   s(t)} 

+  y1(t){-cj)a1(t)y  +  r..(t)}  +  u„(t)  {-(l-<j>)a9(t)y  +  r  (t)}  , 

1       1        i        I  L  I  (n5) 


where 


y±(t) 


=  0     for  x.  >  0  , 

l 


£  0     for  x.  =  0  . 

l 
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Equations  (93)  through  (97)  still  hold  with  a  ,  a~  ,  b  ,   and  b 
now  being  functions  of  time. 

On  a  constrained  subarc  on  which  x..  (t)  =  0   (and  x_  >  0) 

dxl 

for     tn    ^  t   ^  t„      the   control   is   determined  by     -; —  =0      so   that 
1  z  at 


rx(t) 
*    Ct)    =  a7C07  f°r      Cl  *  '  *  *2    "  (116) 


The  multiplier  u,(t)   is  determined  by  the  condition  — —  =  0   and 

i  3<p 

hence 


yx(t)  =  ^-^y  {a2(t)p2(t)  -  ai(t)Pl(t)}  .         (117) 

The  condition  that  u,(t)  ^  0  yields  that  on  a  constrained  subarc 
we  must  have 


ai(t)(-Pl(t))  ^  a2(t)(-P2(t))  .  (118) 

Similar  to  what  we  did  in  [26] ,  let  us  now  consider  the  sPecial  case 
when  a.(t)  =  k  g(t)   for   i  =  1,2.   Then  (117)  becomes 

X  3.  . 

i 

hw  -r^-W  •  (119) 

ai     2        x 

Differentiating  (119)  with  resPect  to  time  and  combining  with  (93) 
and  (94) ,  we  find  that 


Ux(t)  =  ~^y   {a2(t)b2(t)  -  a1(t)b1(t)}  ,  (120) 


E-41 


a1(t)b1(t)  ^  a2(t)b2(t)  ,  (121) 

since  again  p^(t)  ^  0.   The  corner  conditions  (102)  through  (105) 
still  hold  only  with  a..   and  a„  being  functions  of  time. 

(3)   Several  Target  Types. 
In  [26]  (reproduced  in  this  report  as  Appendix  C)  we  first 
considered  a  scenario  with  one  weapon  system  in  battle  against  n 
target  types.   Our  previous  treatment  ignored  the  non-negativity 
restrictions  on  the  force  levels.   To  rectify  this  gap  we  consider 


n 
maximize   {vy(T)   -  £  w.x.(T)}   with  T  specified 
4».(t)  1-1  X   X 

dx. 

subject  to:   - —  =  -<b  .a.y    for   i  =  1 , .  .  .  ,n  , 
J         dt      l  l 


%  -  -  l  Vi  • 


i=l 


x,  , .  .  .  ,x  ,y  k  0  ,    ()) .  ^  0   for   i  =  1 , .  . .  ,n  , 


n 
and    I     d>'  =  1  .  (122) 

i-1   X 


We  focus  on  the  development  of  necessary  conditions  of  opti- 
mality.   To  avoid  being  encumbered  by  too  many  symbols,  we  will  con- 
sider the  case  when  the  X  -  force  is  driven  to  zero  and  assume 

that  x„,...,  x  >  0.   In  this  case,  however,  we  must  consider  a 
I  n 
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slightly  different  form  for  the  Hamiltonian.   (See  pp.  108-109  in 
[8]  for  a  discussion  of  the  difference  between  Hamiltonians  denoted 
there  as  H   and  H  ;  the  Hamiltonian  given  by  (123)  is  H  in  the 
notation  of  [8].   As  pointed  out  in  [8],  whether  one  uses  a  Hamil- 
tonian of  the  form  (92)  or  (123)  is  a  matter  of  taste.) 


n  n 

H(t,xi,P.^i)  =  y{  I      a^-p.)^.}  -  pn+1  I      b.x. 

i-1  i=l 


ii  n 

+  A(l  -  I     4».)  +  yCH^y)  +  I      y±$± 
i=l  i=l 


(123) 


where 


M(t) 


and 


Yi 


=  0     for  x   >  0 


^  0     for  x.  =  0  , 


=  0     for  <{>.  >  0  , 
l 


0     for  <j>.  =  0  . 
i 


The  adjoint  system  of  differential  equations  for  the  dual  variables 


xs 


dt 
and 


dP± 

'  "  b-,-P^i     for  i  -  l,...,n  ,  (124) 


dpn+l  n 

"dT"  =  *lal(pl"V  +  I     ViPi  *  (125) 

i=2 
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On  a  constrained  subarc  on  which  xn(t)  =  0   (and  x~,...,x 

i  I  n 

* 

>  0)   for   t,  <>   t  £  t_,   part  of  the  control  is  clearly  <J>   (t)  =  0. 

To  determine  the  multiplier   u(t),  we  must  take  a  slightly  differ- 
ent approach  than  previously  used  above.   This  development  will  show 
the  reader  the  origin  of  the  form  of  the  Hamiltonian  (123). 

Following  Gamkrelidze  (see  pp.  265-267  of  [22]),  on  a  con- 
strained subarc  the  maximum  principle  takes  the  following  form 


maximize     h(t  ,x  .  ,p.  ,<f> .) 


subject  to:    \     <{>_.  =  1  , 
1=1 


n 

i 


where 


d> .  ^  0       for   i  =  1, .  .  .  ,n  , 
i 


dx1 

-^   =  -^y  ^  0  ,  (126) 


n  n 

h(t,x.  ,p.  ,<j>.)  =  y{  T  a.  (-p.  )<|>.}  -  p     V   b  x   . 
x   l   l      .^,  x   x   x     n+1  .t*1   x  x 
x=l  x=l 


(127) 


We  re-write  (126)  in  a  more  convenient  form  to  apply  the  well-known 
Kuhn-Tucker  conditions  [19].  (Since  the  constraints  are  linear,  it 
is  well-known  that  the  Kuhn-Tucker  constraint  qualification  [19]  is 
satisfied.   See  [2]  for  a  further  discussion.) 
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maximize     h(t,x.,p.,<(>.) 
♦l 


n 
subject  to:    £   <j> .  =  1  , 
i=l   1 


-<J> .  ^  0   for  i  =  1, .  . .  ,n  , 
*1a1y  <:  0  ,  (128) 


Then  the  Lagrangian  for  the  optimization  problem  (128)  on  the  con- 
strained subarc  (which  turns  out  to  be  the  Hamiltonian  (123))  is 
given  by 

n 
L(*1,y,Y1,v±)  =  h(t,xi,pi,<j.i)  +  X(l-  I  <b±) 

i=l 


+  y(-(},1a1y)  +  £   y  ±*  ±  (129) 

i=l 


=  H(t,xi,pi,(j>i)  . 


Applying  the  well-known  Kuhn-Tucker  conditions  [19]  to  (128),  we 
have  that  necessary  and  sufficient  conditions  for  the  maximum  to 
the  concave  program  (128)  are 


3L 

3*-, 


=  -P1a1y  -  A   -  ua^  +  y±  -  0   ,  (130) 


3L 

3<}>.  ri~iy  '  i 


=  -p.a.y  -  X  +  y.    =  0     for      i  =  2,...,n    ,  (131) 


y(c))1*a1y)    =  0    ,  (132) 
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y.<j>.   =  0   for   i  =  l,...,n  ,  (133) 

and 

y,Y.  ^  0.  (134) 

Let  us  assume  that  there  is  an  isolated  maximum  to  (128) 
(thereby  ignoring  the  troublesome  case  of  alternate  optima,  which 
merely  presents  inessential  complexities;  it  is  readily  shown  that 
(128)  cannot  have  alternate  optima  except  at  isolated  points  in  time) 
Then  we  let   i  =  J  be  the  index  such  that  <j)   =1.   By  the  comple- 
mentary slackness  relationship  (133),  we  have  y     =  0  so  that  (131) 


yields 


X   =  ("PJ)aJ  y.  (135) 


Using  (135)  we  may  re-write  the  n  equations  (130)  and  (131)  as 

y  a.  y   -  y     =  y(aJPJ-a;LP1)  ,  (136) 

and 

Y.  =  y{aT(-pT(t)-a.(-p.(t))}   for   i  =  2,...,n.       (137) 

Assuming  that  y  >  0   (otherwise  there  is  no  allocation  problem) ,  the 
Kuhn-Tucker  optimality  condition  y.  ^  0   for   i  =  l,...,n  yields 

a  (-p  (t))  £  a  (-p  (t))   for   i=2,...,n.        (138) 

In  fact,  for  the  maximum  to  be  isolated,  we  must  have 


a  (-p  (t))  >  a  (-p  (t))   for   i  =  2,...,n  and   i  +   J.   (139) 
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We  are  therefore  left  with  one  equation  for  the  two  remaining 
under termined  multipliers   u  and  y 

M  axy  -  Y-l  ■  y^jPj-3!?!)  ■  (14°) 

Let  us  note  that  once  we  reach  the  constrained  subarc  and  x1(t=t1)  ■  0, 
x  (t)   remains  equal  to  zero  for   t..  S  t  S  t.  s  T,   since  the  model 
(122)  does  not  allow  for  replacements.   Thus  by  Gamkrelidze's  device 
the  constraint  |  a  y  ^  0   is  active  for   t..  <  t  ^  T.   Observing  that 
the  constraint   fay  ^  0  being  active  guarantees  that  the  constraint 
<(>..  ^  0   is  satisfied  (we  note  that   a  >  0  and  y  >  0)  ,   we  set 
Y-.  =  0   to  obtain 

"(t)  "  ^  <Vj-Vi>  ■  (1A1) 

Then  the  Kuhn-Tucker  optimality  condition  u  ^  0   implies  that 

a1(-p1(t))  ^  aj(-Pj(t)),  (142) 

on  the  constrained  subarc.   Differentiating  (141)  and  combining  the 
result  with  the  adjoint  equations  (124) ,  we  obtain 

Pn+l(t) 
p(t>  .  -E±L_  (a^-a^)  ,  (143) 

so  that  Gamkrelidze's  condition  (9)  is  only  satisfied  on  a  constrained 
subarc  with  x  =  0  when 

aibi  *  ajbj.  <144> 

since  it  is  readily  shown  that   p   . (t)  >  0   for  0  £  t  <  T. 
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Thus,  we  have  proved  the  following  theorem: 

Theorem  E.l:   For  the  problem  (122) ,  an  optimal 
policy  has  the  property  that  if  we  annihilate 
X..  ,   then  we  must  have 


albl  *  aJbJ  ' 


where  J   is  the  index  of  any  target  type  upon 
which  fire  is  later  concentrated. 

Although  the  above  does  not  complete  the  analysis  of  this 
problem,  we  end  our  discussion  here.   We  propose  to  ONR  that  the 
complete  discussion  of  the  conditions  under  which  it  is  optimal  to 
drive  a  force  level  to  zero  be  a  future  research  task. 
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Appendix  F.   The  Isbell-Marlow  Problem  Revisited. 

1.   Introduction. 

We  have  included  in  this  report  as  Appendix  A  some  of  our 
previous  work  [9]  on  the  Isbell-Marlow  fire  programming  problem  [6] 
(the  same  results  are  apparently  given  in  [7]).  This  work  [9]  revises 
some  even  earlier  work  (see  Appendix  A  of  [8])  that  we  had  done  on 
this  problem.   Since  the  writing  of  [9]  in  June  1971,  we  have  uncovered 
a  gap  in  our  development  of  the  synthesis  of  the  optimal  control  (how- 
ever, the  solution  presented  in  [9]  remains  valid).   (By  the  synthesis 
of  optimal  control,  we  mean  using  the  basic  necessary  conditions  of 
optimality  to  explicitly  determine  the  time  history  of  the  optimal 
control  from  initial  to  terminal  time  as  a  function  of  the  initial 
state  of  the  system.)   Thus,  it  is  the  purpose  of  this  appendix  to 
revise  our  earlier  work  [9]  (repreduced  in  this  report  as  Appendix  A). 
The  results  we  obtain  from  this  further  examination  of  the  Isbell- 
Marlow  problem  are  of  considerable  interest,  since  they  provide  an 
understanding  of  solution  phenomena  (multiple  extremals  and  a  dominated 
payoff)  that  we  have  encountered  in  a  differential  game  [10] . 

In  [9]  we  showed  that  extremals  were  not  unique  for  a  certain 
range  of  model  parameters  and  tried  to  develop  necessary  and  suffi- 
cient conditions  for  optimal  paths  to  lead  to  all  the  terminal  states 
of  combat.   (By  an  extremal  we  mean  a  battle  trajectory  on  which  the 
necessary  conditions  of  optimality  are  almost  everywhere  satisfied  in 
time.)   In  the  pioneering  1956  work  of  Isbell  and  Marlow  [6]  (the 
Pontryagin  maximum  principle  was  only  announced  in  1956  [1],  although 
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Professor  Magnus  Hestenes  of  the  USA  had  apparently  first  given  the 
now  standard  control  formulation  (as  well  as  developed  optimality 
conditions)  and  urged  others  to  follow  his  approach  in  1949  [3])  one 
can  find  several  gaps  (boundary  conditions  for  dual  variables,  treat- 
ment of  state  variable  inequality  constraints  (SVIC))  in  the  light 
of  subsequent  control  theory  developments.   In  [9]  we  followed  Isbell 
and  Marlow's  heuristic  treatment  of  the  determination  of  boundary 
conditions  for  the  dual  variables  and  did  not  consider  the  theory  of 
SVIC.   We  rectify  these  gaps  here. 

Furthermore,  we  feel  that  one  of  our  major  contributions  in 
[9]  was  the  development  of  a  purely  algebraic  method  for  the  deter- 
mination of  the  optimal  control,  as  opposed  to  the  geometric  method 
employed  in  [6] .   The  reader  should  note  the  difficulty  of  applying 
their  geometric  approach  to  a  similar  problem  with  a  state  space  of 
dimension  greater  than  three.   We  feel  that  this  is  essentially 
impossible,  whereas  our  algebraic  method  (like  algebraic  geometry  in 
n-dimensional  space)  appears  to  be  readily  applicable  to  such  problems 

2.   Statement  of  the  Isbell-Marlow  Fire  Programming  Problem. 

For  the  reader's  convenience  we  re-state  the  problem  originally 
studied  by  Isbell  and  Marlow  [6] 
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maximize  {ry(T)  -  px  (T)  -  qx  (T) }  with  T  unspecified, 
<Kt)  Z 

dxl 
subject  to:  ^—  =  -  (fra^y, 

dx 

■  =  -(l-<j))a0y, 


dt     v  Y/  2J 

of  =  "  blXl  "  b2X2'  (1) 

x  ,x  ,y  £  0  and  0  £  <j>  ^  1, 

where  all  symbols  are  defined  in  the  next  section. 

The  battle  terminates  upon  reaching  the  terminal  states  defined 
by  (1)   x  (T)  =  x  (T)  =  0  and  (2)   y(T)  =  0.   Upon  further  analysis, 
it  has  been  convenient  to  consider  that  there  are  the  following  five 
"target  sets"  for  this  problem: 


C±:      Xl(T)  =  0,   x2(T)  >  0,  y(T)  =  0; 


C2:   X;L(T)  =  0  before  x2(T)  =  0,   y(T)  >  0, 


C3:   X;L(T)  =  0  after   x2(T)  =  0,   y(T)  >  0, 


C4:  x±(T)    >   0,   x2(T)  =  0,   y(T)  =  0, 

C5:  x±(T)    >    0,   x2(T)  >  0,   y(T)  =  0. 

The  reader  should  note  that  in  the  above  problem  statement  T   is 
referred  to  as  being  undetermined.   This  is  because  T   is  determined 
by  entry  to  one  of  the  above  five  target  sets.   This  is  a  function  of 
the  control  applied,  and  hence  before  an  allocation  rule  is  given, 
it  is  unspecified. 
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3.   Notation. 

The  symbols  which  are  used  in  this  appendix  are  defined  as 

follows: 

A  =  A(R,z)  =  [z2(R-l)  -  R]/(z-l)2, 

B  =  B(R,z)  =  A(z-l)2/z2  =  [z2(R-l)  -  R]/z2, 

a  ,a  ,b  ,b  =  constant  attrition-rate  coefficients, 

C.   for   i  =  1,2,3,4,5  =  the  i —  part  of  the  terminal  surface 

as  defined  in  section  2, 

D(C.)  =  domain  of  controllability  for  C, 

i  J  i 

g(P  ,R,z)  =  term  in  equation  of  the  locus  of  points  for 
which  P   =  P  , 

h(P  ,R,z)  =  term  in  equation  for  boundary  surface  between  the 

regions  from  which  optimal  paths  lead  to  C.   and  C ,  , 

p,q,r  =  utilities  assigned  to  surviving  X  ,   X   and  Y 
forces  respectively, 

p.(t)   for   i  =  1,2,3  =  dual  variable  corresponding  to 
1  x.(t)  (x3(t)  =  y(t)), 

P.   for   i  =  1,2,3,4,5  =  payoff  associated  with  an  extremal 

leading  to  C . , 

P  =  (x  ,x  ,y  )  =  point  in  the  initial  state  space, 

Q  =  (-p2(t=T))/q 

R  =  a1b1/(a2b2), 

s  =  s(x1,x2)  =  b1x1  +  b2x2, 

t  =  time  at  which  X   is  annihilated,  i.e.   x  (t  )  =  0, 

t„  =  first  time  at  which  2b  x  (tjx.  +  b_(x  )2  =  a  y2(t~)   for  an 
extremal  leading  to  C, , 

T  =  total  time  for  the  battle, 

v  =  v(x)  =  a2p2(i)  -  a^^p^^d), 
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w  =   cosh    /a  b      t    (C   )    =         . 

2    2      14  P0(t= 


a,  (blP2(t=T)+b2p) 


2,.T)  (a^-a^)       • 

x    ,x    ,y  =  average   force   strengths;   with   initial  values     x    ,      x    , 

h    JTTT       (r   ^        ^        (blq"b2P)  R  -  6 

z  =  cosh   ^  ti(C5)    =  T    J^Z^f  =  F^T' 

6   =   a1p/(a2q) , 

4>  =  fraction  of  Y-f ire  directed  at  X  , 

x  =  "backwards  time"  from  the  end  of  the  battle  defined  by 

=  T  -  t,  i.e.  the  time  remaining  before  the  end  of  the  battle, 

t..(C.)  =  "backwards  time"  of  the  first  switch  in  tactics  for 
extremals  leading  to  C . . 


Additionally: 

C   refers  to  C    (the  5 —  part  of  the  terminal  surface)  which 
is  reached  by  extremals  with  a  switch  in  tactics, 

S  S 

P.  =  payoff  associated  with  an  extremal  leading  to  C-. 


4.   Summary  of  Solution. 

There  are  four  cases  to  be  considered 

(1)  6  £  1, 

(2)  R  -  /R(R-l)  <  6  <  1, 

(3)  6  =  R  -  /R(R-l) , 


(4)    0  s;  6  <  R  -  /R(R-l)  , 

where   6  =  a  p/(a  q).   For  Case  (1):   6^1,   the  solution  is  given 
in  Table  I  of  Appendix  A.   Our  re-esamination  here  changes  no  details 
of  the  analysis  presented  there. 
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The  results  of  our  additional  analysis  (which  includes  use  of 
the  theory  of  SVIC)  are  presented  in  Tables  I  through  V.   Supporting 
details  will  be  presented  in  subsequent  sections  only  when  they  differ 
from  our  previous  analysis  (see  Appendix  A) . 

For  Case  (2) :   R  -  /R(R-l)  <  6  <  1,   the  solution  is  shown  in 
Table  I.   Although  this  solution  presented  in  Table  I  (Appendix  F)  is 
identical  to  that  presented  in  Table  II  of  Appendix  A,  there  is  a 
subtle  difference  in  how  it  was  obtained:  our  newer  analysis  (which 
invokes  some  new  results  from  the  theory  of  SVIC  about  the  boundary 
conditions  for  the  dual  variables)  has  revealed  that  the  domains  of 
controllability  do  not  "overlap"  so  that  the  extremals  are  unique. 
Hence,  the  extremal  control  turns  out  to  be  the  optimal  control,  since 
it  may  be  shown  that  the  problem  does  indeed  possess  a  solution. 
Previously  (see  Appendix  A) ,  we  thought  that  the  domain  of  control- 
ability  for  C, ,   denoted  by  D(C,),   had  an  intersection  of  positive 
area  (to  be  more  precise,  positive  measure)  with  other  domains  of 
controllability . 

It  seems  appropriate  at  this  time  to  point  out  to  the  reader 
that  the  optimal  control  has  been  expressed  as  an  open-loop  control. 
For  an  open-loop  control  u  =  u(t;x  , t„)   one  specifies  the  control 
as  a  function  of  time   t  during  the  length  of  the  planning  horizon 
0  :£  t  £  T.   Such  a  control  only  depends  on  the  parameter   t   and  the 
initial  conditions  xn,t„.   This  control  is  not  directly  influenced 
by  the  current  state  of  the  system.   On  the  other  hand,  one  can 
consider  a  closed-loop  (or  feedback)  control  u  =  k(x,t)   which  depends 


F-7 


Table  I.   Solution  to  Isbell-Marlow  Problem  for 


R  -  /R(R-l)  <  6  <  1, 


Nonrestrictive  Assumption:   R  >  1,   i.e.   a  b   >  a  b 


Case  (2):   R  -  /R(R-l)  <  6  <  1  where   6  =  a  p/(a  q) 

Terminal  State  Optimal  Control  Domain  of  Controllability 

*,(tn)   -  0  fl     for     Oitit,  a.b.y^   <:   s2  +   (R-l) (b2x°) 2 


^  s2  +  B(b2x°)2 


!Xl(tl}  =  f  1  for  0  £  t  £  tx        a1b1yQ 

x2(T)  >  0  <j,*(t)  =  j 
y(T)  =0  \   0   for   t   <  t  £  T    aibiyn 

!x   (t   )    =0  (1     for     OstSL 

x2(T)    =  0  <J>*(t)    =  a^y2   >   s2  +    (R-l)(b2x°)2 


y(T)    >   0  v  0      for      t      <    t  £  T 


x1(t2)  >  0  (  1   for  0  £  t  £  t2    a1blyQ  S  R{s2  "  (bixi)2} 

x2(T)  =  0   <|>*(t)  =  | 

y(T)  =0  '  0  for  t2  <  t  <;  T    ^^Q  ^  s2  +  A<b2x2)2 


/x  (T)  >  0  albiy0  *  Rs2{1  "  1/z} 

<  x, 


C   x  (T)  >  0   4>*(t)  =  0   for  0  £  t  ^  T 


y(T)  =  0  albly0  >  R{s2  "  (bixi)2} 


(VT>    >0                          (1  for     0£t£T-Tl  alVo  >   Rs2{1  "  1/z2} 

C^|x2(T)    >    0  <j)*(t)    =     J  albly0   >   S     +  A(b2X2)2 

'     ,m>                                         ^0  for      T-Tn    <    t  £  T  u2           2        uru      °\2 

y(T)    =0                                                          1  albiy0        S          B(Vr 


Definition  of  Times 

(a)  t..   is  first   t   such  that  x  (t,)  =  0. 

(b)  t   is  first   t   such  that   2b  x  (Ox  +  b  (x  ) 2  =  a2y2(t2) 

/— — —      R  —  fi 

(c)  t-   is  determined  by  cosh  /a„b   x,  =  r r  • 

1  Z  Z   1    K  —  ± 
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upon  the  current  state  of  the  system.   The  results  presented  in  [7] 
(see  also  [6])  are  in  the  form  of  a  closed-loop  control.   For  deter- 
ministic systems  (one-sided  problems) ,  it  is  well  known  that  open- 
loop  control  and  closed-loop  control  yield  identical  results  in 
trajectory  and  payoff  [4].   It  is,  of  course,  a  simple  matter  to 
convert  the  optimal  control  presented,  for  example,  in  Table  I  into 
a  closed-loop  control.   However,  for  optimal  control  of  stochastic 
systems  it  is  well  known  that  open-loop  and  closed-loop  controls  do 
not  yield  the  same  return.   In  Appendix  I  we  consider  a  stochastic 
version  of  (1)  and  consider  there  a  closed-loop  control. 

It  also  seems  appropriate  to  note  the  following  for  the 
quantities  A  and  B  which  appear  in  inequalities  defining  various 
domains  of  controllability: 

(a)  for   0  <:  6  <  R  -  /R(R-l)  , 

we  have  A  >  B  >  0,  (2) 

(b)  for   6  =  R  -  /R(R-l) , 

we  have  A  =  B  =  0,  (3) 


(c)   for  R  -  /R(R-l)  <  6  <  1, 

we  have  A  <  B  <  0.  (4) 

Furthermore,  let  us  recall  that  A  and   B  are  defined  by  (see  also 
Section  3) . 

A       A^D     \       R(Z2-1)   -   Z2  ,,v 

A  =  A(R,z)  =  y_^2 ,  (5) 

B  =  B(R,z)  =  R(z21>  -  z2  .  (6) 

z 
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The  quantity   z,   defined  by  z  = -,   relates  A  and  B   to   6 

K  —  J. 


For  Case  (3) :   6  =  R  -  /R(R-l) ,   the  solution  is  shown  in 

Table  II.   The  reader  should  note  the  reduction  in  dimension  (from 

S      S 
three  to  two)  for  the  domain  of  controllability  or  C  ,   D(C  ). 

Case  (3)  is  an  important  case  for  understanding  the  solution  to  the 

Isbell-Marlow  problem.   For  R  -  /R(R-l)  <  6  <  1,   both  extremals  and 

the  optimal  control  are  unique.   In  other  words,  the  domains  of 

controllability  do  not  "overlap."   For   0  £  6  <  R  -  /R(R-l) , 

extremals  are  no  longer  unique  (i.e.  for  a  point  P   in  the  initial 

state  space  there  may  be  more  than  one  extremal  path  leading  to  the  terminal 

surface) .   In  Case  (3) :   6  =  R  -  /R(R-l) ,   extremals  are  unique  for 

S  S 

all  terminal  states  except  C  .   Moreover,  for  C,.,   the  optimal  control 

is  no  longer  unique:   for  any  P  £  D(C,.)   any  policy  as  shown  in 

Table  II  with   t„  <  t..   leads  to  the  same  payoff,   P  =  -qx  /— r— . 

Furthermore,  any  initial  point  P  £  D(C.)   or  D(C.)   or  D(C  )   with 

a  by2  =  s2   also  leads  to  exactly  the  same  payoff  when  an  optimal 

control  is  used,  i.e.  for  such  P   we  have  Pn  =  P.  =  Pc  =  Pc. 

14    5    5 


For  Case  (4) :   0  £  6  <  R  -  /R(R-l) ,   the  domains  of  controlla- 
bility and  corresponding  extremal  controls  are  shown  in  Table  III. 
The  domains  of  controllability  D(C  ),   D(C  ) ,   D(C5),   and  D(C5) 
all  "overlap''  each  other  (as  careful  study  of  this  table  will  show)  . 


(Inequalities  such  as  A(Rsz)  >  B(R,z)   for  0  <;  6  <  R  -  /R(R-l) 
(which  are  essential  in  such  considerations)  are  given  in  Appendix  A.) 
Hence,  step  (e)  of  the  general  solution  procedure  presented  in 
Appendix  A  must  be  applied.   The  resulting  solution  after  this  has 
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Table  II.   Solution  to  Isbell-Marlow  Problem  for 


6  =  R  -  /R(R-l) . 

Nonrestrictive  assumption:   R  >  1,   i.e.   a1D1  >   a9bo 
Case  (3):   6  =  R  -  /R(R-l)   where   6  =  a  p/(a  q) 

Terminal  State         Optimal  Control  Domain  of  Controllability 

/  x1(t1)   =0  [1     for     Oititj         albiy0  <   s2  +   (R"1)(b2x2)2 

C1|  x2(T)    >   0        <j,*(t)    =    J 

(  y(T)    =0  '   0      for      t      <    t  £  T  aib-i7o  a  s2 

!x1(t1)    =0  /   1      for     0  s£  t  £  t 

x2(T)    =  0        o*(t)    =  aibiy2   >   s2  +    (R-i)(b2X°)2 

y(t)    >   0  lo     for     t     <til 

x1(t2)  >  0         (1  for  0^t^t2    aibiy0  *  R{s2  "  (bixi>2} 

C4  iX2(T)  =  °   **(t)  =  I 

y(T)  =0  '  0   for   t2  <  t  £  T    a1b1Yo  *  s2 

j^d)  >  o  aiVo <  R{s2 "  (bixi)2} 

C5     X2(T)    >   °        <t»*(t)    =  0      for      0  <;  t  £  T 

'y(T)  =  o  aibiyo  s  s2 


<  x1(T)    >   0  /l     for     0  s:  t  £  t3  <  t±  a-^^l  =  s2 

^]x2(T)    >   0        <}>*(t)    =     | 

y(T)    =0  (  0      for      t3   <    t  £  T  b2x°   >   blXJ(^  -   1) 


C5  )  ~2 


Definition  of  Times 


(a)  t   is  any   time  such  that   0  £  t„  <  t  , 

(b)  For   t.,   t  ,   and   t  ,   see  Table  I. 
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Table  III.   Extremals  for  Isbell-Marlow  Problem  for 


0  <:  6  <  R  -  /R(R-l) 


Nonrestricitve  assumption:   R  >  1,   i.e.   a.  b   >  a  b 


Case  (4):   0  £  6  <  R  -  /R(R-l)   where   6  =  a  p/(a  q) 

Terminal  State         Extremal  Control         Domain  of  Controllability 
x1(t1)  =0  <   1   for  0  £  t  £  t1  a1h±yl   <  s2  +  (R-D(b2x°): 


„. 


C1  j  x2(T)  >  0    <|>*(t) 

y(T)  =0  0   for   t.  <  t  £  T    a.-b-.y2  >  s2  +   B(b„x")2 


(o 


2'V 


/x  (t  )  =0  (   1   for  0  <;  t  £  t 

C2  |x2(T)  =  0   <|>*(t)  =  albly0  >  s2  +  (R-D(b2x2)2 

(y(T)  >  0-  '  0   for   t   <  t  :£  T 


x  (t  )  >  0  /   1  for  0  £  t  <;  t 


albly0  <  s2  +  (R_1) (b2X2} 
C4|x2(T)  =  0   <J>*(T)  =  albly0  *  R{s2  "  (bixi)2} 

y(T)  =  °  (  0   for   t2  <  t  ^  T    a^y2   ^  g2  +  A(b2xc)2 


(xl(T)  >0  W0<S2+(R"1)(V2)2 

C  ]  x  (T)  >  0   <})*(t)  =0   for  0  £  t  <;  T       albiy0  S  Rs2{1  "  1/z2} 
'y(T)  =  °  a^y2  <  R{s2  -  (b^)2} 


xn  (T)  >  0  /  1   for   0  £  t  £  T-t. 


a^y2  <  s2  +  (R-l)(b2x°)2 


c|!  ]  x  (T)  >  0   <J>*(t)  =  |  albiy0  <  Rs2{1  "  1/z2} 


/ x  u;  >   u        ( 

|x2(T)  >  0    <j>*(t)  = 

'  y(T)  =  °  '  °   f°r   T"T1  <  C  ^  T   a^y2  >  s2  +  A(b2x°)2 

alblY0  >  s2  +  B(b2X2)2 


Definition  of  Times:   for   t1 ,   t„,   and   t  ,   see  Table  I 
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been  done  is  presented  in  Table  IV.   A  major  difference  between  the 

results  presented  here  and  those  of  Appendix  A  is  that  (we  have 

S 
discovered  since  June  1971  that)  extremals  do  lead  to  C,..   However, 

this  gap  in  our  earlier  analysis  (see  Appendix  A)  has  had  no  effect 

upon  the  solution  which  we  presented  earlier,  since  it  may  be  shown 

that  for  an  initial  point  P°  €  D(C  )  f]   D(C  )   we  have  P   :>  P   when 

the  corresponding  extremal  controls  are  used.   Hence,  extremals  lead- 

S 
ing  to  C^  may  be  dropped  (as  they  were  previously  inadvertently 

omitted)  from  further  consideration. 

For  Case  (4) :   0  ^  6  <  R  -  /R(R-l) ,   the  solution  is  shown  in 
Table  IV.   It  is  of  interest  to  note  that  the  solutions  contain  a  dis- 
persal surface  (see  pp.  134-141  in  [5]),  a  solution  aspect  rarely 
encountered  in  optimal  control  problems  (see  [2]  for  the  only  example 
of  which  we  are  aware,  other  than  those  given  in  Isaacs'  book  [5]). 
The  reader  should  note  that  the  solution  shown  in  Table  IV  is  identical 
with  that  given  earlier  (see  Table  II  of  Appendix  A) . 

A  considerable  amount  of  effort  has  gone  into  the  study  of 
extremals  for  0  £  6  <  R  -  /R(R-l) .   In  Table  V  we  show  event  (such 
as  annihilation  of  X  ,   end  of  battle,  etc.)  times  and  payoffs  for 
Case  (4):   0  ^  6  <  R  -  /R(R-l) .   We  omit  derivation  of  these  times, 
which  is  readily  done  using  elementary  considerations  (such  as  combi- 
nation of  the  extremal  control  with  the  solution  to  the  force  level 
equations) .   The  solution  to  the  force  level  equations  is  consequently 
shown  in  Table  VI.   Additionally,  the  reader  should  note  that  if  we 
have  c{>(t)  ■  constant  for  all   t  €  [t..,t  ],   then  the  following 
"generalized  square  law"  holds 
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Table  IV.   Solution  to  Isbell-Marlow  Problem  for 
0  £  6  <  R  -  /R(R-l) . 

Nonrestrictive  assumption:   R  >  1,   i.e.   a  b   >  a  b 
Case  (4):   0  <;  6  <  R  -  /R(R-l)   where   6  =  a  p/(a  q) 

Terminal  State  Region  of  Initial  Force  Levels 

alblY0  <  s2  +  (R-D(b2x2)2 
alblY0  *  Rs2  "  R{b1x°[z2(R-D+R]/(2R)  +  b2x°}2/z2 


x1(t1)  =  0 


C  {   x  (T)  >  0 

y(T)  =0  for   0^x°  <  (b^^/Ckb^ 

alblY0  fe  h(p°>R»z)   for   0  £  x°  <;  kbxx°/b2 


/x1(t1)  =  0 
C2   x2(T)  =  0      a^y2  :■  s2  +  (R-l)(b2x2)2 


y(T)  >  0 


/X;L(t2)  >  0 

)   x_i 


albly0  <  s2  +  (R"1)(b2x2)2 


C4  J  x2(T)  =  0      ^Vo  *  R{s2  "  (bxxi)2> 

*  y(T)  albly0  *  h(p°»R'z)   for  °  *  x2  *  kb!xi/b2 


(xl(T)    >0  «lbli<    s2+    (R-1XV2)2 

C5|x2(T)    >    0  ^Vo   <   R{s2   "    (bixi>2} 

'  Y(T)    =   °  a^y2   <;  Rs2    -  R{blX°[z2(R-l)+R]/(2R)   +  b2x°}2/z: 

for      0  <;  x°  <    (b2x°)/(kb1) 


Explanation  of  Symbols 

(a)  For   t  ,  t?,      and   T-,   see  Table  I 

(b)  k  =  [z2  -  R(z-1)2]/(2R). 
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Table  V.   Event  Times  and  Payoffs  for  Isbell-Marlow  Problem 
when  0  £  6  <  R  -  R(R-l) . 
Terminal  State  C  :   x-jCt-,)  =   °»   x  (T)  >  0,   y(T)  =  0 


h  = 


1        ,    r    /alVo   -  S"+    (b2X2)Z   '  b2X2    , 
ln{  === } 


*Yi 


/a^y0   -  s 


1                    1      /ai  Vn   -  s^  +    (b9x")* 
T  =  t,   +  ■  -  tanh"1!        1  1  ° ^—  } 


^Fi 


b2x°    vfc 


P,    =  — 3—  /sz  +    (R-l)(b  xp*   -  a  by* 
b/R  xxu 


Terminal   State     C    :      x    (t   )    =  0,      x    (T)    =  0,      y(T)    >   0 


1  /a^yg-s^    (b2xQ)^-b2x2 

tx  =  — —  £n{ } 


^1 


^1*1  y0   " 


=   t     +        1        tanh  1{ 


b2x°   /R 


v^b2 


°T^ 


/a^^g   -s^+    (b2xp 


Terminal  State     C^ :      x^t^    =  x^T)    >   0,      x    (T)    =   0,      y(T)    =   0 


for     a  by2   >   s2  ^____ 

110  1              y(t2)   -^(t2)   -yg  +  .^/(«b1) 

t      =  — — — -  £nj  

2  ^ 


y0  -  s/v^" 


for     a  by2   <   s2 


C2   " 


!  /y2(t2)    -  yg  *  s2/(a1b1)    -  y(t2) 

Ant } 


^ 


s/v^-  yQ 


for     a^y2  =  s, 


^  "  ^F  -'  &» 


11 
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Table  V.   (Concluded) 


where 


y(t2)  =  /^  {(2R-l)b2x°  +  2/sz  +  R(R-l)(b2xp^-  a^y^  } 


also 


T  =  t2  + 


x       _±  /  Rb2x°  +  /sz  +  R(R-l)(b2x2)z  -  a^yj 


•V2 


cosh 


(R-l)b2x°  +  /sz  +  R(R-l)(b2xO)z-  a^^y 


P4  =  b^R  ^b2x2(R_:L)  +  ^Z  +  R(R-D(b2x^)z  -  a^y*} 


Terminal  State  C  :   x  (T)  >  0,   x  (T)  >  0,   y(T)  =  0 
(no  switch:  ^>*(t)    =  0  for  0  <;  t  £  T) 


T  = tanh  i  z — o  ■  u — O  J 

/I^-  blXo  +  b2xO 


Terminal  State  C  :  x±(T)    >   0,      x^T)    >   0,   y(T)  =  0 


switch  in  tactics:   <£*(t) 


1     for     0  <;  t  £  T  -  t. 


0      for     T   -  x      <    t  £  T 


1  ,  -1    (R-6^ 

Ti =  7== cosh     feJ 

/a2b2 


T   =   T,    + 


x  (  (/R(zz-1)    -z)  (/a^  yQ+s) 

+  — • £n  <  — ■ 

2/a^bY         (  (/a1b1  yQ-s)  (/R(zz-1)   +   z) 


PS5   -  JL  {-Z(b2x°)  £=t)    +  |  /(zz(R-l)-R)(a1b1y-sO    } 
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Table  VI.   Solution  to  Force  Level  Equations  for 
Two-on-One  Combat . 
dx. 


Combat  Equations 


dt 
dx. 


<(»a1y 


—  =  -(l-<D)a2y 


with        x    (t=0)    =  x    , 


with        x    (t=0)    =  x    , 


dt  =  ~  blXl   "  b2X2        With        y(t=0)    =  ?q- 


Solution:      when     $(t)    =   constant      for      t  €    [0,t    ],      then 


(  (b.x°+bx°) 

r.  n  o  /  11         2      2 

Xl(t)    =  Xl  +  *al)  [(D-b,    +    (l-0)aob 


11 


2   2 


y  cosh/<j>a  b     +    (1-<|>) a  b      t 


(b1x1+b2x2) 


sinh/tba.b,    +    (l-<b)a_b_    t   — : r ; — t; » Z — i 

Aa1b1  +   (l-»)a  b  X   X  22  [*albl  +   <1-*)a2b2] 


o  /  (blX°+b2x2) 

x2(t)    =   x2  +    (1-0) a2j  t  +    (1_<j))  , 


1   1 


2    2 


cosh/ijia  b     +    (l-<J>)a _b      t 


^blXl+b2X2^ 


sinh/^a-b.    +    (l-c|>)a_b_    t   -  7- — r — ,     j\     .. r—\ 

Aa1b1  +    (l-*)a2b2  1X  22  [*albl  +    (1"*)a2b2] 


y(t)    =  y     coshVc^b     +    (!-<}>) a  b      t 


(b1x1+b2x2) 


/<t.a1b1  +    (l-<|.)a2b2 


sinh/^a-.b     +   (H)a  b      t 
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C2(t=t1)  -  C2(t=t2)  =  Ua^  +  (l-4.)a2b2}{y2(t=t1)  -  y2(t=t2)}    (7) 
where 

C(t)  =  blXl(t)  +  b2x2(t). 

This  generalized  square  law  has  been  used  (along  with  the  extremal 
control  to  C.)   to  express  P.  =  ry(T)  -  px  (T)  -  qx_(T)   in  the  form 
shown  in  Table  V. 

As  a  final  check  on  our  theoretical  developments,  we  had  some 
numerical  computations  done  for  Case  (4) .   The  results  shown  in  Table 
V  have  been  used  in  this  work.  We  would  like  to  thank  a  M.S.  thesis 
student,  Robert  L.  Powers,  LT ,  USN,  for  his  efforts  in  this  area. 
All  Powers'  computations  have  supported  the  solution  shown  in  Table 
IV.  For  example,  let  us  examine  the  results  of  computations  for  typical 
data  such  as  that  shown  in  Table  VII.   For  this  data,  we  have  6=0 
and  R  =  2.   Extremals  lead  to  C.  ,   C  ,   and  C,.   from  P  .   For  the 
extremal  to  C.  ,   we  have   t..  =4.26  minutes  while  T  =  18.99  minutes 

and  P  =  -217.72.   For  the  extremal  to  C  ,  we  have  T  =  9.96 

S 
minutes  and  P  =  -149.92.   For  the  extremal  to  C  ,  we  have  T  -  x  = 

S 
3.64  minutes  while  T  =  16.81  minutes  and  P  =  -221.43.   Considering 

Table  IV,  it  is  readily  shown  that  P°  =  (100,100,152)   is  such  that 
the  optimal  path  leads  to  C  ,   as  these  numerical  computations  bear 
out.   Notice  that  for  the  optimization  problem  (1)  which  we  are  con- 
sidering and  the  input  data  shown  in  Table  VII,  making  the  battle  last 
longer  (by  firing  more  at  X.. )   does  not  result  in  more  value  for 
survivors  (from  Y's  standpoint).   Finally,  because  in  our  earlier 
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Table  VII.   Time  History  of  Force  Levels 

g 
for  Extremal  Leading  to  C,. 


x°  =  100,     x°  =  100,     yQ  =  152 


a..  =  0.2   X  -  cas./(min.  x  Y  unit) 


a 


=0.1   X  -  cas./(min.  x  Y  unit) 


b  =  0.1   Y  -  cas./(min.  x  x  unit) 
b_  =  0.2   Y  -  cas./(min.  x  x  unit) 


p  =  0.0,     q  =  5.0,     r  =  10.0 


time,    t 

x1(t) 
100.00 

x2(t) 
100.00 

y(t) 

0.00 

152.00 

0.36 

89.19 

100.00 

144.91 

0.73 

78.88 

100.00 

138.21 

1.09 

69.04 

100.00 

131.88 

1.46 

59.65 

100.00 

125.89 

1.82 

50.69 

100.00 

120.24 

2.19 

42.12 

100.00 

114.91 

2.55 

33.94 

100.00 

109.88 

2.91 

26.11 

100.00 

105.14 

3.28 

18.61 

100.00 

100.69 

3.64 

11.43 

100.00 

96.50 

4.96 

11.43 

88.22 

82.62 

6.28 

11.43 

78.18 

70.18 

7.59 

11.43 

69.69 

58.95 

8.91 

11.43 

62.60 

48.75 

10.23 

11.43 

56.81 

39.40 

11.54 

11.43 

52.20 

30.73 

12.86 

11.43 

48.69 

22.59 

14.18 

11.43 

46.23 

14.85 

15.50 

11.43 

44.77 

7.36 

16.81 

11.43 

44.29 

0.00 
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work  (which  is  reported  in  Appendix  A)  we  had  erroneously  concluded 

o 

that  no  extremals  led  to  C   for  0  a:  6  <  R  -  /R(R-l)  ,   we  also  have 
verified  that  extremals  do  lead  to  C   by  examining  the  time  history 
of  the  force  levels  computed  according  to  the  extremal  control.   An 
example  of  the  results  of  such  computations  is  shown  in  Table  VII. 


5.   Necessary  Conditions  of  Optimality. 

The  solution  shown  in  Tables  I,  II,  and  IV  has  been  developed 
according  to  the  solution  procedure  which  we  outlined  in  Appendix  A. 
This  procedure  is  based  upon  synthesizing  extremal  trajectories  from 
the  basic  necessary  conditions  of  optimality.  Thus,  it  seems  appro- 
priate to  say  a  few  words  about  these. 

Since  we  have  developed  in  Section  4.b.(l)   of  Appendix  E 
the  basic  necessary  conditions  of  optimality  for  (1) ,  we  will  not 
repeat  details  again  but  refer  the  reader  there  for  all  the  details. 
As  we  saw  in  Appendix  E,  the  Isbell-Marlow  problem  requires  use  of 
results  (boundary  conditions  for  the  dual  variables,  necessary  con- 
ditions for  it  to  be  optimal  to  have  x  (t)  =  0   for  a  finite  interval 
of  time,  corner  conditions)  from  the  theory  of  SVIC.   In  the  original 
pioneering  work  of  Isbell  and  Marlow  [6]  these  results  were  not  used, 
since  they  had  not  yet  been  discovered  and  published  in  the  mathe- 
matics literature.   In  our  earlier  work  (see  Appendix  A  in  this  report 
and  also  Appendix  A  of  [8])  we  had  followed  some  heuristic  reasoning 
of  Isbell  and  Marlow  [6]  on  such  points. 

Let  us  now  summarize  the  main  results  of  Section  4.b.(l)   of 
Appendix  E.   We  saw  that  in  order  for  it  to  be  optimal  to  have  x  (t)  =  0 


F-20 


for  a  finite  interval  of  time  we  must  have 


albl  fe  a2b2*  (8) 

At  entry  at   t..   to  a  constrained  subarc  on  which  a  =  0,   we  have 
(from  the  corner  conditions) 

^<0-"iJ*2(V>-  (9) 

p.(t1")  =  P^t^)   for  i  =  2,3,  (10) 

where  t..    denotes  a  left-hand  limit.   Furthermore,  for  C,   we  have 
that 


<  (-p0(t=T))  <:  q.  (11) 


The  latter  of  these  two  inequalities  was  conjectured  in  Appendix  A 
(in  June  1971)  to  be  a  property  of  an  extremal.   Our  recent  work  using 
the  theory  of  SVIC  has  confirmed  this  conjecture. 

S 
6.   Existence  of  Extremals  Leading  to   C  . 

In  our  earlier  work  [9]  we  erroneously  concluded  that  for 

c 

0  £  6  <  R  -  /R(R-l)   there  were  no  extremals  leading  to  C  .   Let  us 
now  retrace  our  earlier  development  in  a  correct  manner. 

There  are  two  subcases  for  entry  to  C^.   We  consider  the 
extremals  for  which 

1   for   0  ^  t  £  T  -  x  , 


>*(t)  =  J 

(  0   fo 


r  T  -  x   <  t  £  T.  (12) 
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For  convenience,  let  us  say  that  such  an  extremal  leads  to  C  . 
Combining  the  above  extremal  control  with  the  generalized  square  law 
(7) ,  we  obtain 

R(b2x2(T))2  +  2Rb1b2x1(T-T1)x2(T)  +  (b^T-T^ ) 2 

+  2(l-R)b1b2x1(T-x1)x2  +  a^y2  -  s2  +  (1-R)(b2x°)2  =  0.    (13) 

A  backwards  integration  of  the  state  equations  of  0  £  t  £  x   with 


x  (t=0)  =  x  (T)   and  y(x=0)  =  0  yields  that 


b2X2  =  b2x2(T)z  +  b1x1(T-T1)(z-1)»  (1Zf) 

since  x7(t=T-T-)  =  x~ .   It  should  be  noted  that  in  deriving  (14)  we 
have  assumed  that  x . (T-t.)  >  0.   (13)  and  (14)  may  be  combined  to 
yield 

(b2x2(T)-b2x2)2  =  (a^yg-s^/A,  (15) 

and  hence  we  require  that  the  right-hand  side  of  (15)  be  positive  in 
order  that  C_   be  reached.   Thus,  for  P  £  0(0,.)   we"  must  have 

a  by2  <  s2   for   A  <  0,  (16) 

aibiyo  =  s2   for  A  =  °'  (17) 

a  by2  >  s2   for   A  >  0,  (18) 

where  the  reader  should  keep  (2)  through  (4)  in  mind. 

Now  when  A  <  0  (corresponding  to  R  -  /R(R-l)  <  6  <  1) ,   (15) 
may  be  solved  to  yield 
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/sz   -   a..b  y^ 
R  -  z4-l)  •         <19> 


b2x2(T)  =■  b2x2  .  ^  w/R  .  giS(IHL) 

The  requirement  that  x  (T)  >  0   then  leads  to 

alblY0  >  s2  +  A(b2x2)2'  (20) 

where  we  have  made  use  of  (5) .   Combination  of  (14)  and  (19)  yields 
that 

Vi(I-V  "  -V2  +  z/r  -\'(r-S  •  (21> 

The  requirement  that  x. (T-x  )  >  0   then  leads  to 

alblY0  <  s2  +  B(b2x2)2j  (22) 

where  we  have  made  use  of  (6) .   Furthermore,  the  time  T-x   may  be 
determined  by  solving  the  following  equation  (see  Table  VI) 


y(T-T.)  =  y.  cosh/a,  bn  (T-x,) —  sinh/aTbT  (T-x,)   to  yield 

1      U        i  1      1     i — r —        11      i 


/a  b  y  (T-x  )  -  a  b  yf.   +  sz  -  /a  b  y(T-x  ) 

T  -  t  =  —  in         L   X  Z - ^^ — —     (23) 

/a^b-  s  -  /a^  yQ 


The  requirement  that  the  quantity  in  brackets  is  greater  than  one  leads 
to 

yo >  y^-V-  (24) 

Now,  the  generalized  square  law  (7)  may  be  combined  with  the  extremal 
control  (12)  and  (14)  to  yield 
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^  ylT-j)  -  ^U^IT  ]\  _'^°    .  (25) 


(24)  and  (25)  may  be  combined  to  yield 


aibiyo  >  Rs2  {1  "  1/z2}-  (26) 

Furthermore,  (23)  and  (25)  may  be  combined  to  yield 

,      (  (z-VR(z^-l))  (s+/aTb~  y  )) 

T  -  T  «  L—   £n  ) ,        }•  (27) 

2/a^bY    (  (s-/a^  y0)(z+/R(Z2-l)  ) 

S 
Finally,  (19)  may  be  used  to  show  that  the  payoff  P_   corresponding 

to  use  of  the  extremal  control  (12)  is  given  by 


ps  -A 

5   b2 


|-z(^)b2x°  +  |  /(R-zz(R-l))(s^-a1b1y^)}  .   (28) 


When  A  >  0   (corresponding  to   0  £  6  <  R  -  /R(R-l)) ,   we  must 
have  a- by2  >  s2  by  (18)  so  that  analysis  similar  to  the  above 
readily  yields  results  similar  to  (20) ,  (22)  ,  and  (26)  but  with  the 
inequalities  reversed  (see  Table  III) .   In  our  previous  work,  we  had 
erroneously  concluded  that  this  case  was  impossible. 

Next,  we  consider  the  extremals  for  which 

4>*(t)  =0  for  0  £  t  £  T.  (29) 

Let  us  adopt  the  terminology  that  such  an  extremal  leads  to  C   with 
payoff  P  .   We  omit  discussion  of  results  previously  derived  in 
Appendix  A.   In  that  appendix  we  showed  that 


Using  (30),  we  may  express  P   in  the  following  form 
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b2x2(T)  =  -  b^°  +  /sz  -  a^yjj/R.  (30) 


P   --i 
5   b2 


{z(^i)blX°  -  /s*  -a^yg/R}.         (31) 


7 .   Proof  of  a  Dominance  Theorem. 

In  this  section  we  show  that  for   0  £  6  <   R  -  /R(R-l)   we  need 
not  consider  extremals  that  lead  to  C  ,   since  the  payoff  associated 
with  such  an  extremal  path  is  exceeded  (or  dominated)  by  the  payoff 
associated  with  an  extremal  leading  to  C  .   Hence,  our  previous  over- 
sight  on  extremals  leading  to  C,.   is  of  no  consequence. 

Our  main  result  is  stated  below  as  Theorem  1.   It  is  convenient, 
however,  to  first  explain  our  terminology  in  a  precise  fashion.   To 
be  precise,  then,  we  have 

Definition  1:   We  say  that  an  extremal  leads  to  C 
when  we  use  the  extremal  control 


4>*(t)  = 


1   for   0  £  t  s;  T  -  t  , 


0   for  T  -  in  <  t  £  T 
1 


and  reach  the  terminal  state  x  (T)  >  0, 
x2(T)  >  0,   y(T)  =  0.   (t-   is  given  in 
Table  V.) 

We  denote  the  payoff  associated  with  an  extremal  leading  to  C,.   by 


P5-   We  have  that  for  0  £   5  <  R  -  /R(R-l) 

P5  =  b-   {-Z^b2x2  +  I  A^CR-D-RHajb^-s*)  }.    (32) 
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g 
Furthermore,  the  domain  of  controllability  for  C  ,   denoted  by 

D(C5) ,   is  given  by 

for  0  £  6  <  R  -  /R(R-l) 

DCCj)  =  {P°  =  (x°,x°,y0)  |  ajb  yg  *  Rs2(l-l/z2)   and 


s2  <  s2  +  B(b2x°)2  <  a^y2  <  s2  +  A(b2x°)2}.       (33) 


We  also  have 


Definition  2:  We  say  that  an  extremal  leads  to  C   when 
we  use  the  extremal  control 


$*(t)  =  0  for  0  £  t  £.   T, 

and  reach  the  terminal  state  x  (T)  >  0, 
x2(T)  >  0,   y(T)  =  0. 

We  denote  the  payoff  associated  with  an  extremal  leading  to  C,.   by 
P  .  We  have  that  for  0  s;  6  <  R  -  /R(R-l) 

Furthermore,  the  domain  of  controllability  for  C  ,   denoted  by  D(C  ), 

is  given  by 

for  0  £  6  <  R  -  /R(R-l) 

D(C5)  =  {P°  =  (x°,x°,yQ)  |  a^y2  £  Rs2(l-l/z2)   and 

a^y2  <  R{s2  -  (b^)2}  }.     (35) 
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To  be  precise,  we  must  also  add  the  following  condition  to  (33)  and 
(35) 

albly0  <  s2  +  (R"1)(b2x2)2'  (36) 

Since  (36)  does  not  enter  into  the  proof  of  Theorem  1,  we  have  (for 
convenience)  omitted  it. 

We  now  state  the  main  result  of  this  section  as  Theorem  1. 

THEOREM  1:   Assume  that   0  ^  6  <  R  -  /R(R-l) .   (This  is 
equivalent  to   z  >  /R/(R-1) .)   Then  P°  = 

(x°,x°,y0)  €  D(C5)  0  D(C5)   implies  that 

S  o 

P  ^  P   with  equality  holding  only  when  a  b  yjr 


Rs2(l-l/z2). 


PROOF : 

A.   First  we  will  show  that 


s2  <  a^y2  <  Rs2(l-l/z2),  (37) 

and 

R 


R-l 


<  zl,  (38) 


Q  r\  Q 

imply  that  P   <  P  .   For  P  €  D(C  )  fl  D(C  )   we  have  (postponing 
for  now  the  case  when  equality  holds) 

aibiyo  <  Rs2(1~1/z2s>>  <39> 

which  implies  that 

R(z2-l)s2  -  z2a  b  y2  >  0.  (40) 
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Squaring  (38) ,  we  may  rearrange  the  result  to  obtain 

(2Rsz/sz  -  a^y^R)2  <  {R(z2+l)s2  -  z2a  b  yg}2.     (41) 

By  observing  that  1  -  1/z2  <  1  +  1/z2  and  considering  (39) ,  it  is 
readily  seen  that 

R(z2+l)s2  -  z2a1b1y2  >  0,  (42) 

so  that  we  may  take  square  roots  in  (41)  to  obtain 

2Rsz/sz  -  a-jb-jyg/R  <  R(z2+l)s2  -  z2a  b  y2.       (43) 

Now  it  is  easily  seen  that  (41)  may  be  rearranged  to  yield 

(z2(R-l)-R)(a1b1y2-s2)  <  {z(R-l)s  -  R/sz  -  a^^/R}2.    (44) 

The  conditions  s2  <  a.b.y2  and  z2  >  R/(R-1)   are  readily  combined 
to  yield 

R2(s2-a1b1y2/R)  <  z2(R-l)2s2.  (45) 

Recalling  that  for  P°  £  D(C-)  f)  D(C5)   we  have  a  b  yQ  <  Rs2(l-l/z2) 
<  Rs  ,  we  may  extract  square  roots  in  (45)  to  obtain 

z(R-l)s  -  R/sz  -  a  b  y2/R  >  0.  (46) 

Considering  (37) ,  (38) ,  and  (46) ,  we  may  extract  square  roots  in 
(44)  to  obtain 

|  /(zz(R-l)-R)(a1b1yz-sz)  <  z  (^)  (b^+b^)  -  /s2  -  a^^g/R.     (47) 
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Some  additional  manipulations  on  (47)  now  readily  yield  that  (37)  and 
(38)  imply  that  P   >  P  . 

B.   Next  we  show  that 


aibiyo  =  Rs^1"1/22).  (48) 

S 
implies  that  P-  =  P-.   It  is  readily  shown  that  (48)  leads  to 

/(z^(R-l)-R)(a1b1y^-s  )  =  ^(z2(R-l)-R)  .        (49) 

Now  (32)  and  (49)  may  be  combined  to  yield 

Furthermore,  (48)  also  leads  to 


ps  --a- 

5   b2 


s2  -  a^yg/R  =  s2/z2,  (51) 

which  may  be  combined  with  (34)  to  yield 

s 

Consideration  of  (50)  and  (52)  shows  that  (48)  implies  that  P_  =  P_ 
By  A.  and  B.  above,  Theorem  1  is  proved.  Q.E.D. 

8.   Development  of   Conditions  for  Extremals  to  Reach  C . . 

Another  topic  for  which  our  previous  discussion  [9]  was 
inadequate  was  the  development  of  conditions  for  extremals  to  reach 
C, .   (Previously,  we  heuristically  developed  (66)  and  had  not  shown 
that  (70)  was  a  necessary  condition.) 
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Let  us  define 


a  (b  p  (t=T)+b  p) 
w  =  w(p„(t=T))  = 


Po(t=T)(a1b1-a2b2)  ' 


which  we  may  write  as 

where 

Q  =  (-p2(t=T))/q.  (54) 

Then  recalling  (11)  (which  was  a  necessary  condition  of  optimality) , 

we  have  that 

for  6  <  Q  <;  1, 

we  have         1  <  w(Q)  £  f^f  =  z,  (55) 

K.-  i 

where  w(Q)   is  a  strictly  increasing  function  of  Q  arid  w(Q=6)  =  1. 
Now 

w  -  cosh/a2b2  (T-t2) ,  (56) 

and 

z  =  cosh/a-b  ?  =  cosh/a  b   T-  (C  )  . 

Since  cosh  x  is  a  strictly  increasing  function,  (55)  and  (56)  readily 
yield 

T  -  t2  £  x1  =  T^).  (57) 


Now,  the  extremal  control  is 
4>*(t)  = 


1  for  0  i  t  <  t,, 


0  for   t  £  t  £  T,  (58) 
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where   t   is  the  smallest   t   such  that 


albly2(t2}  =  RUb1x1(t2)+b2x°)2  -  (blX;L(t2))2}.     (59) 

In  [9]  we  showed  that  the  extremal  control  (58)  and  the  generalized 
square  law  (7)  yield 

blXl(t2)  =  b2x2(R_1)  +  /sZ  +  R(R-D(b2xp^  -  a^y*  .   (60) 
Clearly  we  must  have  b  x  (t_)  £  b  x  ,   and  this  may  be  combined  with 


(60)  to  yield 


b2x°(R-l)  £  bxxj.  (61) 


A  backwards  integration  of  the  state  equations  using  (58)  yields 


for   t_  £  t  £  T 


b2x2(t)    =  b1x1(t2){coshv/a^b2~  (T-t)    -  1},  (62) 


so   that 


b2x2(t2)  =  b2x2  =  blXl(t2)  {cosh/a^  (T-t2)  -  1}.     (63) 


Then  (60)  and  (63)  may  be  combined  to  yield 


T 


T  -  t  =  -— —  cosh   <  >.     (6< 

/I^  (  (R-l)b2x2  +  /sz  +  R(R-l)(b2xpz  -  I^yg") 


Recalling  that 


Tl  =  xl^    =  ~L~  MBh'1^)  ,  (65) 
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we  may  combine  (57)  and  (64)  to  obtain 

a1b1yQ  £  s  +  A(b2x°)2.  (66) 

Now,  since  <$>*(t)    =  1   for  0  £  t  £  t_,   we  have  by  the  generalized 
square  law  (7) 

albly0  "  a1V2(t2)  =  s2  "  (b1x1(t2)+b2x°)2.      (67) 
We  may  combine  (59)  and  (67)  to  obtain 

albly0  =  (R-DUb1x1(t2)+b2x2)2-s2}  +  R{(bxx°)2  -  (blX;L(t2))2} 

+  R{s2  -  (blX°)2}.    (68) 
We  will  prove  as  Proposition  1  that 

R{(bxx°)2  -  (blX;L(t2))2}  -  (R-l){s2  -  (blX;L(t2)+b2x°)2}  ^  0,       (69) 
so  that  (68)  and  (69)  yield 

aibiyo  *  R{s2  "  (bixi)2}*  (70) 

It  remains  to  prove  the  following  proposition 

PROPOSITION  1:   A  necessary  and  (with  appropriate  additional 
assumptions)  sufficient  condition  for  (69) 
to  be  true  is  that  b_x„(R-l)  £  b,x  . 

PROOF:   We  prove  necessity  only  (sufficiency  follows  by  reversing  the 
chain  of  arguments) . 
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It  is  readily  shown  that  (69)  may  be  rearranged  to  yield 

(blX°)2  -  (blX;L(t2))2  ;>  2(R-l)b2X2{blX°  -  b^Ct^)}.    (71) 

Now  (71)  may  be  factored  and  (assuming  that  b  x  >  b  x  (t„))  a  common 
term  cancelled  on  both  sides  to  yield 

blX°  +  blX;L(t2)  £  2(R-l)b2x2  (72) 

Combining  (60)  with  (72) ,  we  obtain 

/s*  +  R(R-l)(b2x°)^  -  a1b1yg  £  b2x°(R-l)  -  b^J,      (73) 

and  the  necessity  of  the  proposition  is  proved.  Q.E.D. 


The  time  t   in  (58)  may  be  explicitly  determined  by  solving 
the  equation  (see  Table  VI) 


y(t=t2)    =  yQ  cosh/a^by  t2 sinh/a  b      E   ,  (74) 


and  this  yields 


t2  =  -  £n^ >  (75) 

v/a1b1         (  yQ   -  s//a1b1  ) 

Since  the  quantity  in  brackets  must  be  k  1   (for  all  cases,   i.e. 
a.b.y2  >  ,   =,   <  s2)  ,   we  must  have 

yQ  ^  y(t2)  >  0.  (76) 

Now  (59)  and  (60)  may  be  combined  to  yield 
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o 

Y2(t2)  =  ~  {(2R-l)b2x°  +  2/s*  +  R(R-l)(b2x2)z  -  ^xVp    ,    (77) 

so  that  when  this  is  combined  with  (76)  ,  we  obtain 

(  a  b  y2  J 

{    b  x°  "  (2R-l)b2x°>  :>  2/s*  +  R(R-l)(b2x°)^  -  a^yg  .    (78) 

Hence,  we  must  have 

albly0  *  R(2R-l)(b2x°)2.  (79) 

If  (79)  holds,  then  squaring  both  sides  of  (78)  again  leads  to  (70). 
Moreover,  (61)  and  (70)  yield  (79)  as  Proposition  2  shows. 

PROPOSITION  2:   Assume  that 


/^  ,  \ ,   o   ,   o 
(R-l)b2x  £  bxx  . 


Then 


a  by2  £  R{s2  -  (b  x  )2}   implies  that 

albiy0  *  R(2R"1)(b2X2)2- 
PROOF:   We  may  manipulate   (R-l)b  x  £.   b  x   to  obtain 

(2R-l)(b2x2)2  £  2blX°b2x2  +  (b2x°)2. 


Hence 


R(2R-l)(b2x2)2  <:  R{s2  -  (blX°)2}, 
and  whence  the  proposition.  Q.E.D 
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9 .   Analysis  of  Dispersal  Surface. 

For  0  £  6  <  R  -  /R(R-l) ,   the  domains  of  controllability 
for  all  the  terminal  states  are  shown  in  Table  III.   It  is  easily 
seen  that  many  of  these  domains  overlap  each  other  (the  inequalities 
(47)  through  (53)  of  Appendix  A  are  useful  in  such  considerations) . 
Hence,  we  must  use  "considerations  in  the  large"  to  select  the  optimal 
policy  from  among  the  candidate  extremal  policies.   This  is  essentially 
step  (e)  of  our  general  solution  procedure  outlined  in  Section  4.  of 
Appendix  A. 

Our  "considerations  in  the  large"  have  led  us  to  discover  a 
new  aspect  of  the  Isbell-Marlow  problem  [6] :   the  solution  contains 
dispersal  surfaces  for  0  £  6  <  R  -  /R(R-l) .   A  dispersal  surface  (see 
pp.  134-141  in  [5])  is  a  surface  only  away  from  which  optimal  paths 
lead.   It  is  the  locus  of  points  such  that  the  payoffs  from  two  families 
of  extremals  are  equal. 

Let  us  now  show  how  the  dispersal  surface  which  separates 
optimal  paths  leading  to  C   and  also  C   is  explicitly  determined. 
We  have  previously  summarized  results  in  [9]  (see  Appendix  A  in  this 
report),  but  we  present  the  details  here. 

It  seems  appropriate  to  firmly  establish  our  notation  first, 
however.   To  be  precise,  then,  we  have 


Definition  3:   We  say  that  an  extremal  leads  to  C   when  we 
use  the  extremal  control 


4>*(t)  = 


1  for  0  £  t  si  t  , 


0   for   t„    t  as  T, 


"1 


and  reach  the  terminal  state  x  (T)  = 
x1(t1)  =  0,   x2(T)  >  0,   y(T)  =  0. 
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We  denote  the  payoff  associated  with  an  extremal  leading  to  C.   by 


P    .      We  have   that 


pi  ■  7^ /sZ  +  (R-1>(VPZ  -  aiVo  • 


(80) 


Furthermore,  the  domain  of  controllability  for  C  ,   denoted  by  D(C.), 
is  given  by 


o  o 


^2 


DC^)  =  IP"  =  (x^,x^y0)  |  a^y2  <  s2  +  (R-l)  (b^) 


and 


alVo  ^  s2  +  B(b2x2)2}-     (81) 


We  refer  the  reader  to  Section  7  for  the  corresponding  notation 

S 
and  results  for  C^  and  C^.   However,  let  us  be  more  precise  in 

(35) ;  we  now  write  this  as 

for  0  £  6  <  R  -  /R(R-l) 


D(C5)  =  {P°  =  (x°,x°,y0) 


albly0  <  s2  +  (R"1)(b2x2)2» 


aJj-y*  s:  Rs2(l-l/z2),  and 


aiVo  <  R{s2  "  (bixi>2}-}  •   <82> 


Considering  (33) ,  (35)  (or  equivalently  (82) ,  and  (81) ,  we  see 

c 
that  D(C  )  0  D(C  )  0  D(C5)   is  nonempty  (see  inequalities  (47) 

through  (53)  of  Appendix  A).   However,  Theorem  1  tells  us  that  we  need 


not  consider  further  extremals  leading  to  C  .   Thus,  we  must  determine 
optimal  paths  from  extremals  for  P  6  D(C  )  f|  D(C-)  . 


Now 


by  (81)  and  (82)   P°   such  that  a  b  yg  =  s2  +  Btb^)2 


clearly  belongs  to  D(C  )  0  D(C  ).   (The  reader  should  recall  (2): 
B  >  0.)   We  begin  by  stating  and  proving  Theorem  2. 
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THEOREM  2:   Assume  that   0  £  6  <  R  -  /R(R-l) .   Then  for 
albiy0  =  s2  +  B^b2X2)2'   We  haVe  P5  >  Pl* 


PROOF : 

A.   Let  us  first  observe  that   6  <  R  -  /R(R-l)   implies  that 


z2  > 


LR-lj 


(83) 


B.   Now  it  is  readily  shown  that  (83)  leads  to 


'R-l 


z2(  R ^)(blxl)2  +  2blXIb2X2  *  S"  "  (b2X2)2' 


(84) 


We  may  add  the  term   (b~x  )2/z2   to  both  sides  of  (84).   This  makes 
the  left-hand-side  of  (84)  a  perfect  square  so  that  extracting  square 
roots,  we  obtain 


(¥Kxi  +  ^r  >  /s2-!<s2  +  B(V2>2>> 


(85) 


where  we  have  made  use  of  the  definition  (6)  of  B.   Recalling  that 
we  are  considering  the  case  when  a  by2  =  s2  +  B(b  x  )2,  we  see 


that  (85)  leads  to 


'R-l 


z^b^  -/s<  -a1b1y^/R>  - 


b2X2 


(86) 


Hence,  considering  (34)  for  P  ,   we  have 

a  (b2x2} 


P5> 


(87) 
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C.   For  a-fe-y^  =  s2  +  B(b  x  )2,   it  is  readily  shown  by  (80)  that 

P..   reduces  to 

0  <b2*2> 
P1.--A-iS-.  (88) 

whence  the  theorem  follows.  Q.E.D. 


Now  for   0^6  <  R  -  /R(R-l) ,   we  showed  in  Appendix  A  that 
s2  +  B(b2x°)2  <Rs2(l-l/z2).  (89) 

Hence,   P   such  that  a  by2  =  Rs2(l-l/z2)   clearly  belongs  to 


D(C  )  0  D(C  ) .   We  now  state  and  prove  Theorem  3. 


THEOREM  3:   Assume  that   0  £  6  <R  -  /R(R-l)  .   Then  for 

albiy0  =  Rs2(1"1/z2)   and  p°  €  ^V  H  D(C5) 
we  have  P±   >   P5» 


PROOF : 


A.   Again  we  observe  that   6  <  R  -  /R(R-l)   implies  that 


2    '  R 


B.   We  may  readily  obtain  from  (90)  that 

z2(^)(blX°)2  >  (s-b2x°)2,  (91) 

which  is  easily  further  manipulated  upon  to  yield 

{f  "  Z^bix?}2  >   r  {s"  +  CR-D(V2)2  -  R82^-1/22)^-    (92) 

Now  for   6^0,   we  readily  have 
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.       o  rR-l 

bixi  * z  he 


By  (82)  we  have  that  for  P°  6  D(C  ) 


)blX°.  (93) 


alblY0    <R{s2   "    (b!xi)2}-  <94) 


Since  we  are  assuming  that 


aibiyo  =  Rs^1"1/22).  (95> 

we  may  obtain  from  (94)  that 

so  that  (93)  and  (96)  yield  that 

Considering  (97) ,  we  may  extract  square  roots  in  (92)  to  obtain 


\  =  z(^)blX°  >  4:^  +  (R-l)(b2x£)*  -  Rs  (1-1/z*).   (98) 

/  R 

Recalling  the  expression  (80)  for  P  ,   we  readily  see  from  (98) 


that 


C.   Since  we  are  assuming  that  (95)  holds,  it  is  readily  shown 


using  (34)  that  P   reduces  to 


p    =  -*- 

5   b2 


M¥M-f}-  <100> 


whence  the  theorem  follows.  Q.E.D 
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Thus,  by  Theorems  2  and  3  we  have  shown  that  for  0  s:  6  <R  -  /R(R-l) 
(or,  equivalently,  for  /R/(R-1)  <z  £  R/(R-1)) 
A.   P5  >  P1   for  a^y2  =  s2  +  B(b2x°)2, 


or 


and 


or 


(P1-P5)  <  0 


B.   P1  >  P5   for  a^y2  =  Rs2(l-l/z2), 


(P1-P5)  >  0 


where  we  also  assume  that  P   6  D(C.)  f)  D(C  )  .   We  also  note  that 
s2  +  B(b2x°)2  <Rs2(l-l/z2)   for  0  <;  6  <R  -VR(R-l).   The  locus  of 
points  for  which  P  =  P    (or  P  -  P  =  0)   is  the  dispersal  surface. 
It  is  uniquely  determined,  since  we  always  have 


at  (Pi"P5)  >  °' 


(101) 


We  now  prove  (101)  as  Theorem  4. 


THEOREM  4:   Assume  that  x  >  0  and  R  >  1.   Then 


(P  -P_)  >  0. 


ay0   i  5 


PROOF : 

A.      Using    (34)    and    (80) ,    it   is   easily   computed   that 


aiVoq 


-2—  (P   -P   ) 


b2v/R 


J             M  +    (R-l)(b2x2)^   -  a^y*  | 
1   "/ Rs2   -   a,b   v2  ( 


i"iy6 


1 


•  s*  +    (R-l)(b2xp^    -  a^yg 


(102) 
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B. 


Now  for  x  >  0,   we  have 


(R-l)(bx°)2  <  (R-l)s2  for  R  >  1 


(103) 


We  may  rearrange  (103)  and  perform  further  manipulations  to  obtain 


's*  +  (R-l)(b2xp*  -  a^yg 

^   -   Wo        <1§ 


(104) 


whence  the  theorem  follows. 


Q.E.D. 


Our  results  (Theorems  2,  3,  and  4)  are  summarized  in  Table  VIII 
The  dispersal  surface  is  the  locus  of  points  such  that  P  =  P  . 
Equating  (34)  and  (80) ,  we  find  the  equation  of  the  dispersal  surface 


as 


1-5 


albly0  =  Rs2  "  Rtb1x°[z2(R-l)  +  R]/(2R)  +  b2x°}2/z2,    (105) 


for 


0  £  x°    <2Rb2x°/{b1[z2   -  R(z-l)2]}. 


(106) 


Expression  (106)  must  also  be  considered,  since  (94)  must  also  hold 
for  extremals  leading  to  C,..   Considering  Theorem  4  and  (105)  (with 
(106)),  we  readily  obtain  the  corresponding  results  shown  in  Table  IV. 


For  0^6  <R  - /R(R-l)  ,   there  may  also  be  a  dispersal  surface 


separating  optimal  paths  which  lead  to  C   and  C , .  As  noted  in 

Appendix  A,  the  details  are  much  messier  than  those  for  S     ,   and 

1-5 
an  explicit  expression  has  not  been  obtained  for   S 


1-4 
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Table  VIII.   Summary  of  Results  on  Dispersal  Surface 
which  Separates  Optimal  Paths  Leading  to  C   and  C  . 

For  0  <;  6  <  R  -  /R(R-l)    (i.e.   for  /=—  <  z  £  ~r) 

r       R— 1  R— 1 


1.   (P1"P5)  <0       for   a^y2  =  s2  +  B(b2x°)2 


2.   (P1-P5)  >  0       for   a^y2  =  Rs2(l-l/z2) 


3.  ~—  (Pt-Pc)  >  0   for   x°  >  0  and  R  >  1 
dy0 


Let  us  note  that      s2  +  B(b  x°)2  < Rs2(l-l/z2) . 


Then  1,  2,  and  3  imply  that  there  exists  a  uniquely  determined  surface 

such  that  P   =  P  .   Denote  this  surface  as  S 

1    3  Dl-5 


Clearly  S     C  {D(C  )  0  D(C  )}. 

1-5 
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10.   Proof  of  a  Useful  Inequality. 

The  following  lemma  has  been  useful  in  developing  the  domains 
of  controllability  for  C   and  C^. 

LEMMA  1:   Assume  that   0  £  y  ^  1  and  x  ^  1.   Then  a  necessary 
and  sufficient  condition  for 

tanh   y  ^  cosh   x    to  hold  is  that 

2    x2-l 
PROOF:   We  recall  that 


and 


tanh  X  J  =\   An(£3  ,  (107) 


cosh  -1"  x  =  £n(x+/xz-l)  .  (108) 


A.   Proof  of  necessity: 

-1         -1 
Assuming  that   tanh   y  ^  cosh   x,   we  may  use  (107)  and 

(108)  to  obtain 

*n  (3?)  *  JUUx+v'x2^!)2.  (109) 

Since  the  logarighm  is  a  strictly  increasing  function,  (109)  yields 
that 

1  +  y^-  ;>  2x2  +  Ixv^T.  (110) 

Further  manipulation  on  (110)  lead  to 

YT"  -  x2  ^  xv^=T.  (Ill) 
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Since  the  right-hand-side  of  (111)  is  ^   0,   we  may  square  both  sides 
to  obtain 

1        2x2        O  fS   1  o\ 

-p. r-j-  -  ■= ^  -xz.  (112) 

(l-y)z   1-y 

Now  since  1  -  y  ^  0,  we  may  multiply  both  sides  of  (112)  without 
changing  the  inequality.  Rearranging  the  resulting  expression,  we 
readily  obtain 

y2  *  4^  •  (ii3) 

Thus,   tanh   y  ^  cosh   x  implies  that  (113)  holds. 


B.   Proof  of  sufficiency: 

x2-l 
Assuming  that  y  ^  — j—  and  recalling  that  0  £  y  £  1,  we 


readily  see  that 


y  >   y2  :>  ^   .  (114) 


x' 


Writing  this  as 

a  straightforward  chain  of  manipulations  leads  us  to 

(t^-  -  x2)  :>  {xv^I}2.  (116) 

Combining  (114)  and  (115)  ,  we  see  that 

-±-  -   x2  :>  0,  (117) 

1-y 

so  that  we  may  extract  square  roots  in  (116)  to  obtain 

-^-  -  x2  ^  xv/x^T.  (118) 

i-y 
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Straightforward  manipulations  upon  (118)  lead  us  to 

/P^  *  x  +  /^l, 
v    1-y 

whence  by  (107)  and  (108)  we  obtain 


(119) 


tanh   y  ^  cosh   x.  (120) 


x2-l 


Thus,   y2  :> — J—  implies  that  (120)  holds.  Q.E.D, 


Let  us  now  illustrate  the  use  of  Lemma  1.   Consideration  of  (29) 
yields  that  T  =  T(C  )   is  given  by 


1       -1  (  /*2^2  Y0  ) 
T  =  — -^—  tanh  1  {      Z  Z  U  >  .  (121) 

/qq         (     s     J 

s 

We  also  have  that   t,  =  t,  (C,.)   is  given  by 

T,  =  cosh   z.  (122) 

Now  C   is  reached  from  P   when  T(C  )  £  t  (C  ) ,   since  in  both 
cases  we  use  <j>*(t)  =  0  for  the  final  portion  of  the  battle.  Applying 
Lemma  1,  we  readily  find  that 

a-^y2  £  Rs  (1-1/z2).  (123) 

Lemma  1  is  also  used  to  determine  the  condition 

albly0  *  s2  +  B(b2X2)2>  (124) 

for  D(C1). 
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11.   Discussion. 

For  a  discussion  of  the  structure  of  the  optimal  allocation 
policies,  the  reader  is  directed  to  Section  6  of  Appendix  A.   Here  we 
will  try  to  put  into  perspective  what  we  have  learned  from  this,  our 
latest,  examination  of  the  Isbell-Marlow  problem. 

First  of  all,  we  have  concluded  that  the  theory  of  SVIC  is 
absolutely  essential  for  solving  such  problems.  Many  of  the  results 
in  the  pioneering  1956  work  of  Isbell  and  Marlow  [6]  (see  also  [7]) 
are  unsupported  by  such  analysis  (which  uses  theory  developed  since 
1956).   When  more  than  two  target  types  are  considered  (i.e.   n  versus 
one  combat) ,  the  theory  of  SVIC  must  be  used  to  determine  the  order 
in  which  it  is  optimal  to  annihilate  target  types.   This  was  not 
essential  for  the  simplest  case  of  n  =  2. 

Secondly,  the  algebraic  method  (see  Section  4  of  Appendix  A) 
that  we  have  developed  for  the  synthesis  of  optimal  policies  appears 
to  us  as  the  only  way  to  treat  such  problems.   It  should  suffice  for 
us  to  point  out  that  Isbell  and  Marlow  [6]  (see  also  [7])  failed  to 
discover  the  dispersal  surfaces  in  the  fire  programming  problem's 
solution  when  they  used  their  geometric  approach.   Additionally,  we 
do  not  feel  that  this  geometric  approach  can  be  readily  extended  to 
problems  with  n  target  types  (n  >  2) ,   whereas  our  algebraic  approach 
apparently  can. 

Our  re-examination  of  the  Isbell-Marlow  problem  has  been 
particularly  fruitful,  since  it  revealed  the  dominated  payoff  for 
0  <:  5  <  R  -  /R(R-l)  ,   i.e.   P°  €  D(C  )  fl  D(C  )   implies  that  P5  >  P5< 
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We  had  previously  observed  such  a  phenomenon  in  the  supporting  weapon 
system  game  of  H.  K.  Weiss  [11]  (see  [10]  and  also  Appendix  B  of  [8]). 
At  that  time,  we  could  not  relate  such  a  dominated  payoff  in  a  differ- 
ential game  to  any  corresponding  phenomenon  in  the  simpler  one-sided 
(optimal  control)  case.   (It  should  be  noted  that  Weiss  did  not  use 
the  subsequently  well-known  "in  the  small"  necessary  conditions  of 
optimality  from  generalized  control  theory  [4]  (i.e.  optimal  control/ 
differential  game  theory)  to  develop  his  solution  in  [11]  (it  was 
erroneously  reported  that  he  did  by  S.  Sternberg  in  [7]).) 

Finally,  we  would  like  to  point  out  that  our  work  here  lays  a 
firm  foundation  for  considering  two-sided  problems.   We  propose  this 
to  ONR  as  a  future  research  task.   It  should  be  noted  that  Sternberg's 
work  [7]  does  not  cite  any  of  the  essential  theory  of  SVIC.   In 
Appendix  E  we  have  pointed  out  that  lack  of  consideration  of  the  theory 
of  SVIC  is  a  gap  in  the  current  theory  of  differential  games  (particu- 
larly as  applied  to  tactical  allocation  problems  in  the  Lanchester 
theory  of  combat) . 
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Appendix  G.   The  Prescribed  Duration  Battle. 

1.   Introduction. 

One  of  the  objectives  of  this  research  sponsored  by  ONR  is  to 
investigate  the  sensitivity  of  the  optimal  target  selection  policy  to 
the  form  of  the  combat  attrition  model.   Of  particular  interest  is 
whether  for  the  simplest  target  selection  problem  the  structure  of  the 
optimal  policy  is  sensitive  to  whether  combat  attrition  is  modelled 
as  a  deterministic  or  a  stochastic  process.   In  Appendix  I  we  consider 
the  optimal  control  of  the  Lanchester  stochastic  process  for  a  pre- 
scribed duration  battle  between  a  homogeneous  Y-force  and  a  heteroge- 
neous enemy,  X  and  X  .   In  order  to  compare  the  structures  of  the 
optimal  target  selection  policy  for  deterministic  and  stochastic  attri- 
tion processes,  it  is,  of  course,  necessary  to  develop  a  solution  to 
the  deterministic  problem.   We  do  that  in  this  appendix. 

We  have  previously  considered  (several  versions  of)  the  pre- 
scribed duration  battle  [3],  [4]  (reproduced  in  this  report  as 
Appendices  B  and  C) .   Our  subsequent  examination  here  has  led  to  a 
revision  of  one  of  our  earlier  tentative  conclusions  about  the 
structure  of  the  optimal  target  selection  policy:  we  now  conclude 
that  when  target  types  undergo  a  "square-law"  attrition  process 
(as  we  consider  here)  the  optimal  allocation  may  depend  (indirectly) 
upon  the  force  levels.   We  have  reached  this  conclusion  after  a 
careful  study  of  the  problem  at  hand. 
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Also  of  interest  as  regards  the  sensitivity  of  the  struc- 
ture of  the  optimal  policy  is  whether  the  nature  of  the  scenario 
(i.e.  planning  horizon)  affects  the  structure.   In  [3]  we  compare 
the  optimal  target  selection  policy  for  the  prescribed  duration 
battle  with  that  for  a  terminal  control  battle  (a  battle  which 
only  ends  when  force  levels  have  reached  some  specified  terminal 
state).   After  our  re-examination  here,  we  again  will  do  this. 

In  view  of  the  extensive  details  on  basic  necessary  condi- 
tions of  optimality  and  synthesis  of  the  optimal  policy  for  simi- 
lar problems  available  elsewhere  in  this  report  (see,  in  particular, 
Appendices  A,  E,  and  F) ,  we  will  take  the  liberty  of  outlining  some 
results  here.   As  it  is,  our  documentation  of  the  problem's  solution 
is  fairly  lengthy.   We  may  possibly  present  complete  details  in  a 
future  report. 

Finally,  we  note  that  a  key  aspect  in  our  obtaining  these 
new  results  has  been  our  use  of  the  theory  of  state  variable  in- 
equality constraints  (SVIC) .   The  reader  can  find  a  fairly  extensive 
discussion  of  this  theory  and  its  application  to  target  selection 
problems  in  the  Lanchester  theory  of  combat  in  Appendix  E. 
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2.   The  Optimal  Control  Problem. 
We  consider  the  problem 

maximize   {ry(T)  -  px^T)  -  qx2(T)}  with  T   specified, 

dxl 
subject  to:       =  -<£a  y, 

dx. 

"  =  -(l-<j>)a0y, 


dt     v  T/  2 

■^=-Vrb2x2'  (1) 

x1,x2>y  ^  0,  Os^l,  and  T  £  T  , 

where  all  symbols  are  defined  in  the  next  section. 

The  battle  lasts  for  0  ^  t  £  T   unless,  of  course,  one 
side  or  the  other  is  annihilated  before  T..  .   To  be  more  precise, 
the  battle  terminates  under  one  of  the  following  three  conditions: 


(1)  X;L(T)  =  x2(T)  =  0  and  T  <;  T]_ , 

(2)  y(T)  =  0   and   T  £  T± , 

(3)  T  =  T1> 

where  T  denotes  the  time  at  which  the  battle  ends.   Upon  further 
analysis,  it  has  been  convenient  to  consider  that  there  are  eight 
"terminal  states,"  or  "target  sets."   These  are  shown  in  Table  I. 
The  reader  should  note  that  for   S,   through   SR   the  battle  ends 
when  the  force  levels  reach  given  terminal  conditions.   For  these 
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Table  I.   Definition  of  Terminal  States  for 
Prescribed  Duration  Battle. 


S±:      Xl(T)  >  0,   x2(T)  >  0,   y(T)  >  0,   T  =  T 


S2:  X;L(T)  =  x1(t1)  =  0,  x2(T)  >  0,  y(T)  >  0,   T  =  T± 

where  t  <  T 


S3:  x±(T)   =  x1(t3)  >  0,   x2(T)  =  0,   y(T)  >  0,   T  «  T± 

where  t  <  T 


S4:   X;L(T)  >  0,   x2(T)  >  0,   y(T)  =0,   T  £  T± 


S5:   Xl(T)  =  x1(t1)  =  0,  X2(T)  >  0,  y(T)  =0,   T  £  T± 

where  t  <  T 


S6:   Xl(T)  =  X;L(t2)  >  0,   X2(T)  =  0,   y(T)  =0,   T  £  T± 

where  t  <  T 


S?:   X]L(T)  =  x1(t1)  =  0,   X2(T)  =  0,   y(T)  >  0,   T  <;  T± 

where  t  <  T 


S8:   Xl(T)  =  0,  x2(T)   =   X2(t4)  =  0,   y(T)  >  T± 

where   t   <  T 
4 
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terminal  states,   T  is  undetermined,  since  it  is  determined  by  entry 
to  the  terminal  state,  and  this  depends  upon  the  control  used.   For 
these  cases,  the  well-known  transversality  condition  must  hold. 

3.   Notation. 

The  symbols  which  are  used  in  this  appendix  are  defined  as 
follows: 

A  =  A(R,z)  =  [z2(R-l)  -  R]/(z-l)2, 

B  =  B(R,z)  =  A(z-l)2/z2  =  [z2(R-l)  -  R]/z2, 

a  ,a_,b  ,b  =  constant  attrition-rate  coefficients, 

D(S.)  =  domain  of  controllability  for  S^ 

p,q,r  =  utilities  assigned  to  surviving  3L,   X2  and  Y 
forces  respectively, 

p.(t)   for   i  =  1,2,3  =  dual  variable  corresponding  to  x^t) 

(x3(t)  =  y(t)), 

P   for   i  =  1,...,  8  =  payoff  associated  with  extremal 
i 

leading  to   S. , 


o  o 


P  -  (xj\x"y „)  =  point  in  the  initial  state  space, 


R  =  a1b1/(a2b2), 

t  Vi 
S   for   i  =  1,.. . ,8  =  the  i —  part  of  the  terminal 
i 

surface  as  defined  in  Table  I, 
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s  =  s(x°,x°)  =  bxx°  +  b  x°, 


t   =  time  at  which  X   is  annihilated,   i.e.   x-(t-)  =  °> 

t?  =  first  time  at  which  2b  x  (t  )x°  +  b  (x2)2  ■  a2y2(t2)   for 
an  extremal  leading  to   S , , 

D 


t   =  last  time  at  which  fire  is  directed  at  X.   for  an 
extremal  leading  to   S„, 


t,  =  time  at  which  X   is  annihilated  (before  X  ),  i.e. 
x  (t.)  =  0,   for  an  extremal  leading  to   SQ , 

2.       4  o 


T  =  time  at  which  battle  ends, 


T  =  Maximum  possible  duration  for  bat  Lie,  i.e.  T  ^  T  , 


T  =  lower  bound  on  T  (developed  from  condition  x~ (T)  =  0) 
for  an  extremal  leading  to   S„, 


v  =  v(t)  =  a2p2(T)  -  a1P1(T), 

x1 ,x„,y  =  average  force  strengths;  with  initial  values  x°,x°,y  , 

R-6 


=  COSh   v^  Tl  (S4)  =  ~% 


r 
a  =  — 


q  v    a2 


6  =  a1p/(a2q), 


<J)  =  fraction  of  Y-f ire  directed  at  X  , 
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x  =  "backwards  time"  from  the  end  of  the  battle  defined  by 
t  =  T-t,  i.e.  the  time  remaining  before  the  end  of  the 
battle, 


t.CS.)  =   "backwards  time"  of  the  first  switch  in  tactics  for 
extremals  leading  to   S., 

Additionally,  remarks  similar  to  those  for  t1(S.)   above  apply  to 
t1(Si),  T(S±),  etc. 

4.   Outline  of  Solution  Procedure. 

Before  giving  our  solution  algoritm  which  is  based  upon 

Pontryagin's  maximum  principle*  [2],  it  seems  appropriate  to  define 

some  terms.   We  have  then 

Definition  1:   By  an  extremal  path  we  mean  a  path 

on  which  the  necessary  conditions  of 
optimality  are  everywhere  satisfied 
(we  use  the  word  everywhere ,  since  we 
assume  that  the  class  of  admissible 
control  functions  is  the  class  of 
piecewise-continuous  functions). 

Definition  2:   By  extremal  control  we  mean  the  con- 
trol used  in  order  that  the  system 
follow  an  extremal  path. 

Definition  3:   By  the  domain  of  controllability  of  a 
given  terminal  state  we  mean  that  sub- 
set of  the  initial  state  space  from 
which  extremals  lead  to  the  terminal 
state. 


We  use  the  form  commonly  used  in  this  country.   There  is  a  sign 
difference  between  developments  in  the  United  States  and  those  in  the 
Soviet  Union  (see  p.  108  of  [1]). 
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Definition  4:   By  the  synthesis  of  the  optimal 

control,  we  mean  the  explicit  deter- 
mination of  the  time  history  of  the 
optimal  control  from  initial  to  ter- 
minal time  (possibly  as  a  function  of 
initial  conditions) . 

Our  solution  algorithm  then  is  as  follows: 

(a)  extremal  control  is  determined  by  maximizing  the  Hamiltonian; 
since  the  state  variables  (force  strengths)  are  non-negative, 
the  control  depends,  in  many  cases,  only  on  relationships 
between  the  dual  variables  (marginal  return  from  destroying 
targets) , 

(b)  from  each  separate  terminal  state,  the  time  history  of  the 
dual  variables  is  obtained  by  a  backward  integration  of  the 
adjoint  system  of  differential  equations  combined  with  the 
extremal  control;  for  a  square  law  attrition  process,  the 
adjoint  equations  are  independent  of  the  state  variables; 
the  corner  conditions  must  be  considered  in  this  step, 

(c)  for  each  terminal  state  the  domain  of  controllability  is 
determined  by  forward  integration  of  the  state  equations 
using  the  time  history  of  extremal  control  developed  in 
(b) ;  changes  in  control  with  time  (existence  of  transition 
surface)  may  have  to  be  considered  in  this  step. 

(d)  the  solution  is  determined  at  this  point  for  regions  of 
the  initial  state  space  which  are  covered  by  part  of  the 
domain  of  controllability  for  only  one  terminal  state; 
one  must  also  verify  that  the  entire  initial  state  space 
has  been  accounted  for,  since  otherwise  one  may  have  over- 
looked some  type  of  "singular"  surface  or  even  a  terminal 
state, 

(e)  if  domains  of  controllability  overlap  so  that  for  a  point 
of  £he  initial  state  space  contained  in  their  intersection 
there  is  more  than  one  extremal  leading  to  the  terminal 
surface,  then  one  computes  the  payoff  associated  with  each 
extremal;  the  optimal  trajectory  is  selected  from  the  ex- 
tremals by  comparing  these  values. 

The  above  solution  algorithm  is  essentially  the  same  as  that 

presented  in  Section  4  of  Appendix  A.   Let  us  make  a  few  remarks 

about  the  application  of  this  procedure  to  the  prescribed  duration 

battle.   In  this  case,  we  may  think  of  time  as  being  an  additional 
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state  variable.   On  the  other  hand,  in  the  Isbell-Marlow  terminal 
control  problem,  time  may  be  considered  to  be  a  parameter  and  con- 
sequently was  eliminated  from  the  determinations  in  step  (c)  above. 
In  other  words,  in  the  Isbell-Marlow  problem  a  domain  of  controlla- 
bility was  determined  by  inequalities  involving  the  three  state 
variables;  in  the  prescribed  duration  battle  such  a  determination 
as  in  step  (c)  above  involves  the  four  variables  T,  x  ,  x  ,  and  y~  . 
For  the  prescribed  duration  battle  we  have  not  obtained  (relatively) 
simple  analytic  expressions  in  step  (c)  of  the  above  procedure. 
Consequently,  we  could  not  use  analytic  methods  to  accomplish  steps 
(d)  and  (e).   In  other  words,  we  have  not  been  able  to  analytically 
determine  when  the  domains  of  controllability  corresponding  to  dif- 
ferent terminal  states  "overlap."  Hence,  in  many  cases  we  have  ana- 
lytically only  been  able  to  determine  in  extremal  control. 

However,  we  can  use  computational  methods  to  determine  the 
optimal  control.   We  have  expressed  our  "solution"  to  this  problem 
(presented  in  the  next  section)  so  that  we  can  readily  to  the  follow- 
ing: given  a  point  P°  =  (x  ,  x  ,  yQ )   in  the  initial  state  space 

and  T   we  can  determine  which  terminal  states  are  reached  by  ex- 
1    f 

tremals.   In  other  words,  we  can  determine  to  which  terminal  states 
are  reached  by  extremals.   In  other  words,   we  can  determine  to 
which  domains  of  controllability  P°  belongs.   Then,  using  the 
extremal  control,  we  can  compute  numerically  the  payoff  associated 
with  each  extremal  and  select  the  optimal  policy  from  a  finite 
number  of  possibilities.   A  M.S.  thesis  student  (Robert  L.  Powers, 
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Lt.,  USN)  is  developing  a  digital  computer  program  to  use  the 
"solution"  of  Tables  II  and  III  to  determine  the  optimal  target 
selection  policy.   Using  this  computer  program,  we  plan  to  examine 
how  much  difference  there  is  between  payoffs  from  using  optimal 
and  non-optimal  policies. 

5 .   Summary  of  Solution. 

We  have  applied  our  solution  procedure  of  Section  4  to 
develop  a  "solution"  in  the  sense  discussed  there.   Without  loss  of 
generality,  we  may  assume  that  a  b1  >  a?b    i.e.   R  >  1.   Then, 
there  are  two  cases  to  be  considered 

(1)   0  ;>  1, 

and   (2)   0  <;  6  <  1, 

where  5  =  a  p/(a2q). 

For  Case  (1):   6  2:  1>   the  solution  is  shown  in  Tables  II. 
In  these  tables,  the  domain  of  controllability  and  optimal  policy 
is  given  for  each  terminal  state.   Since  the  domains  of  controlla- 
bility don't  overlap,  the  extremals  are  unique.   Since  it  may  be 
shown  that  a  solution  exists  for  this  optimization  problem,  the 
unique  extremal  control  is  consequently  the  optimal  control.   More- 
over, the  optimal  policy  may  be  expressed  in  a  particularly  simple 
form:  always  concentrate  all  fire  on  X1   while  x  >  0.   We  have 
presented  complete  details  on  the  domain  of  controllability  for  each 
terminal  state  and  given  all  "event"  time  for  two  reasons:  (1)  to  be 
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Table  II.   Solution  to  Prescribed 

Duration  Battle  for   6^1. 


Nonrestrictive  assumption:   R  >  1,   i.e.   a  b  >  a  b 


Case  (1):   6^1  where   6  =  a-p/(a  q) 

1    2 

S1:      X;L(T)    >   0,   x2(T)    >   0,   y(T)    >   0,    T  =  1 
Optimal  Control:      <J)*(t)   =   1     for  0  £  t  £  T 


Case   1.      a  b  y  2    > 


;  3- 

T1  < In 


iVo2"^  +(V°2)2  "   V°2 


/*?!     (  J*K  yo  ' s 

Case   2.      a  b  y2   <    s2 

Subcase  A.      a1b1yQ2    £  s2    -    (b2xo)2 


b0x°  -  Ja.b^2    -s2    +(b2x°)2 

T,    <  £n 


1  \     "2  2        \     1T0 


J*K  \  S  "  S*^  y0 

Subcase   B.      a  b  y  2    <   s2    -  /b»x    )2 


T     <  tanh 


7v7 


Case  3.      a  b  y  2    =   s2 

1   1  0 


1 

T,    <  Jin 


^/a-b- 


o 
!"2 
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Table  II.   Solution  to  Prescribed 

Duration  Battle  for  5^1. 
(cont.)  -  1 


S2:   xx(T)  =  x1(t]_)  =  0,  x2(T)  >  0,  y(T)  >  0,   T  =  T± 

where  t   <  T 


Optimal  Control:   <J>*  (t)  = 


1   for   0  <£  t  £  t    where  x  (t  )  =  0 
0   for   t   <  t  <;  T 


2  ° 

Always  must  have   a  by   >  s2  -  (b„x  )2 

Case  1.   a  b  yQ2  ^  s2  +  (R-l) (b2x°)2 


Case  2 


t,  <;  T„  <  t,  + 


tanh 


-1 


1  ~     1    1    / — — 
\/a2b2 

a^^2  £  s  +  (R-l)(b2x°) 


b^  V1T 


v/alV02^2  +  (b2X2    )2     J 


t,    £  T.,    <   t_  ■+ 


11  1        / — Z 

Va9b 


tanh 


-1 


rv/a^yQ2   -  s2  +   (b^)2 


2   2 


v°  vr 


where  t.  is  given  by 


(1)   for  a1b1y02  >  s2 


t,  = 


1    « 


In 


/, 


iVo2   "  '*     +      (b2X2    >2   "  V2°      1 


^a^,      y     -   s 
11     ;0 


(2)      for      anb,y  2    <   s2 

11;0 


r    b„x 


fcl   = 


In 


/Vi 


2  2 


_     / 


Wo2"s2+    (b2X2) 


2    . 


s   -    /a^    yQ 


Table  II.   Solution  to  Prescribed 


5_ :   (concluded) 


Duration  Battle  for   6  ;>  1. 
(cont.)  -  2 


(3)   for  a^y  2  -  s2 

1      (   s 

tn  =  Jin 


^1*1    I  b2X2 


S3:  Xl(T)  -  Xl(t3)  >  0,  x2(T)  =  0,  y(T)  >  0,  T  =  T± 

where  t_  <  T 


NO  EXTREMALS  LEAD  TO  THIS  END  STATE 


S4:  xx(T)  >  0,  x2(T)  >  0,  y(T)  -  0,  T  as  ^ 

ptimal  Control:   <)>*(t)  =  1  for  0  <.   t  <.   T 

in  of  Controllability:   a1b1yo2  <  s2  ~  ^b2x2^2 


oma 


i^A    yo 

tanh     <J }^T1 
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Table  II.   Solution  to  Prescribed 

Duration  Battle  for   6  ^  1. 
(cont.)  -3 


S5:   X;L(T)  -  J^O^)  =  0,  x2(T)  >  0,  y(T)  =  0,  T  £  T 


where  t  <  T 


Optimal  Control:   <}>*(t)  = 


1   for   0  s£  t  £  t-   where  x  (t  )  =  0 


0   for   t   <  t  £  T 


Domain  of  Controllability:   a  b  y  2  ^  s2  -  (b„x_)2 

albly02  <  s2  +  (R-D(b2x2)2 


T  -  t-  + 


^?2 


tanh 


-i  \/IFi 


b2x2°    V^T 


£  T, 


where  t   is  given  by 

(1)   for  a^y  2  >  s2 


t,  = 


in  \  — 


^  ^1  l 


^i  y0  _s 


(2)   for  a1b1yQ2  <  s2 


t,    = 


1    ^S         I 


i    V°2      "   ViV   -  S?  +   <b2^2)2 


s   -    /a1b1     yQ 


(3)      for     a^^2  =   s2 


I. 


1 

in 

V°2 


1    ^       I 


Table  II.   Solution  to  Prescribed 

Duration  Battle  for   6^1. 
(concluded)  -4 


S6:  xx(T)  =  x±(t2)   >  0,  x2(T)  =  0,  y(T)  =  0,  T "*  T 


where   t„  <   T 


NO  EXTREMALS  LEAD  TO  THIS  END  STATE 


S?:   X;L(T)  =  x1(t1)  =  0,   x2(T)  =  0,   y(T)  >  0,   T  £  T 

where   t   <  T 

(  1  for  0  £  t  £  t-   where  x  (t  )  =  0 
Optimal  Control:   <J>*(t)  = 


(  0   for 


"1 
Domain  of  Controllability:   a  b  y  2  >  s2  +  (R-l)(b  x°)2 


b2*2   *" 
T  =  tn  +  tanh  " 


) 


^F2        J  ViV  -  ■' +  M)2   )    1 


where   t..   is  given  by 


(   «VlV   -=^<V2)Z-  b2X2°  ) 


t1  =  £n 

^A  ^?1  yo  "  s 


Sg:   Xl(T)  -  0,   x2(T)  =  x2(t4)  =  0,   y(T)  >  0,   T  £  T± 


where   t,  <  T 

4 


NO  EXTREMALS  LEAD  TO  THIS  END  STATE 
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able  to  make  numerical  comparisons  with  the  stochastic  control  prob- 
lem considered  in  Appendix  I,  and  (2)  to  be  able  to  perform  a  sensi- 
tivity analysis  on  target  selection  policies. 

For  Case  (2):   0^5  <  1,   the  solution  is  shown  in  Tables  III. 
It  is  expressed  in  the  same  format  as  for  Case  (1)  with  the  exception 
that  the  controls  are  extremal  (with  two  exceptions  noted  below) . 

It  should  be  recalled  that  for  the  Isbell-Marlow  terminal  con- 
trol problem  (see  Appendix  F)  when  0  ^  6  <  R  -VR(R-l)   some  domains 
of  controllability  overlapped.   Hence,  extremals  were  not  unique  and 
considerations  "in  the  large"  had  to  be  used  to  determine  the  optimal 
policy.   We  suspect  that  similar  behavior  happens  in  the  prescribed 
duration  battle,  since  we  may  consider  the  Isbell-Marlow  problem  to 
be  imbedded  in  the  prescribed  duration  battle  (i.e.  when  T  <  T  , 
the  solution  to  the  Isbell-Marlow  problem  is  applicable).   Moreover, 
lack  of  explicit  analytic  expressions  for  two  domains  of  controlla- 
bility (that  for   S   when  there  is  a  switch  in  tactics  and   S„) 
has  prevented  explicit  determination  as  to  which  overlap.   Further- 
more, the  computation  of  the  payoff  associated  with  each  (overlap- 
ping) extremal  has  not  been  analytically  tractable.   Hence,  (except 
for  two  instances)  extremal  controls  and  (either  implicit  or  ex- 
plicit) expressions  for  the  corresponding  domains  of  controllabil- 
ity are  given  in  Tables  III.   However,  as  discussed  in  Section  4 
above,  one  can  use  the  information  presented  in  Tables  III  to 
numerically  determine  an  optimal  target  selection  policy  for  any 
specific  set  of  input  values. 
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Table  III.   Solution  to  Prescribed 

Duration  Battle  for  0^6  <  1, 


Nonrestrictive  assumption:   R  >  1,   i.e.   a  b   >  a  b 


Case  (2):   0  <i   6  <  1  where  6  =  a  p/(a  q) 


S±:     Xl(T)  >  0,  x2(T)  >  0,  y(T)  >  0,  T  =  T± 


Since  y(T)  >  0,  the  switching  time   t  =  T- (S. )   is  given  by 


\" 


^Fz 


In 


+     /zz  +  az   -  1 


1  +  a 


b2 

where   z  =  — ; —  and  a  =  — „ 


R  -  1 


q  \  a, 


Case  1.   t  2:   T 


Extremal  Control:   <f>*  (t)  =  0  for  0  <;  t  <;  T 


Subcase  A.   a  b  ,yn  <   R 


s2  -  (blX°)2 


Ti< 


tanh 


^2 


Subcase  B.   a^.y.2  ^  R 


(1)  for  a^y^  > 


r  vaTbT 

1  1     y( 

s/r~~ 


s2  -  (blX°)2 


*»2 


I  < 


■VI 


In 


\\yj  -  Msz   ~   N^FT     "  blX;L0   /FT' 

*/a1hl  yQ  "  s  v^— 


Table  III.   Solution  to  Prescribed 


Duration  Battle  for  0  £  <$  <  1. 
(cont.)-l 


S. :   (continued) 


(2)   for  a  by  2  <  Rs2 


rbnx°  JR  -   /a  b  y  2  -  R{sz  -  (b^V)2) 
11        l  i  0  1  ly 


i  < 


VI 


Jin 


s^R~  -  ^b'1  yQ 


(3)      for     a1b1yQ2  =  Rs2 


T,    < 


/a„b„ 
2  2 


Jin 


b2x2 


Case  2.      t     <   T 


Extremal  Control:   <f>*(t)  = 


1      for     0  <.  t  £  T  -t. 


0      for      T  -   x      <    t  £  T 


Require   that      bjX    (T  -  T.)    >   0,   b  x    (T)    >   0,      y(T)    >   0. 


where 


b-jX    (T-t)    =   s   cosh/a-jb-LCT— r,)    -   /a-jb.,    yQ   sinh/a^.  (T-t    )    -  ^>2X2 


b  x  (T)  ={s  cosh/a  b  (T-t  )  -  /a  b  yQ   sinh/a  b  (T-t  ) }  (cosh/a^  T^-l) 


-/a  b  {y  cosh/a  b  (T-t,) sinh/a  b  (T-T^lsinh/a^  Tj  +  ^2X2' 


^57 
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Table  III.   Solution  to  Prescribed 


S..  :   (concluded) 


Duration  Battle  for  0^6  <  1. 
(cont.)~2 


y(T)  =  {yQ  cosh  a1b1(T-T1)  -  — -   sinh  ^X"(T-t^}  cosh  v£7b7  r 


^A 


-|  s  cosh  /a^bY  (T-Tx)  -  v^b^  yQ  sinh  /^(T-tA5™™*1   2    ^ 


^2 


S2:   X;L(T)  =  x1(t1) 


Extremal  Control:   <t>*(t)  = 


0,   x2(T)  >  0,  y(T)  >  0,   T  =  T 
where  t   <  T 
1  for  0  £  t  £  t.   where  x-(t-) 


=  0 


0  for   t   <  t  £  T 


Always  must  have  a  b  y  2  >  s2  -  (b_x  ) 


fcl  +  Tl  *  Tl 


Case  1.   a^^2  ;>  s2  +  (R-l)(b2x°)2 


T  <  t  + 
1    1 


S*?2 


tanh 


-1 


b2x2°v¥ 


Case  2.   a^y^  <  s2  +  (R-l)(b2x°)2 


T  <  t   + 
1    1 


/a„b„ 
2  2 


tanh 


-1 


'  /aibiy0z  "  sl  +^b2x2)2   ' 


b2x2  /T 


where   t,  =  t1(S  )   is  given  by 


z  +  /^r?~nr ' 


Ti = 


^2^ 


In   ! 


1  +  a 


Table  III.   Solution  to  Prescribed 


V 


(concluded) 


and  z  = 


R-6 

R-l  '   a  "  q/ 


Duration  Battle  for   0  <;  <$  <  1, 
(cont.)-3 


and  where   t..  =  t  (S  )   is  given  by 
(1)   for  a^y^  >  s2 


Cl  = 


si^l 


In 


Valbly0Z"   S'Z  +    (b2X°2)2"  "  b2X2 


o     a 


^X    yo  "  s 


(2)      for     ajb  yQ2   <   s2 


t,    = 


1    ^7 


In 


'b2X2      "   SVo"   "  8*  +   (b2X2} 


S  "  v^ibi    yo 


(3)      for     a^^2   =   s2 
1 


t,    = 


v^bT 


in 


b„x' 


2    2 


S3:      X;L(T)   =  X;L(t3)    >   0,      x2(T)   =  0,      y(T)    >   0,      T  =   ^ 


1     for     0  <;  t  <;  t, 


Extremal  Control:      <j>*  (t)   = 


where   t   <   T 

where  explicit  expression 
for  t„  is  given  below. 


0  for   t   <  t  <;  T 


Always  must  have  a  by2  ^  R  {s2  -  (b  x  )2} 


t„  ^  T  -  T-  (see  below) 


Case  1.   a^^2  <  s2  +  (R-l)(b2x°)2 


TQ  *  T1  <   T(S6) 
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Table  III.   Solution  to  Prescribed 

Duration  Battle  for  0  <;  5  <  1 
(cont. )-4 


S_:   (continued) 


where  T_   is  given  below 


T(S6)  =  t2(S6)+-^ 
^2*2 


-1 


'  Rb2X2  +  &   +R(R-i)(b2X°)2  -a^2 


(R-l)b2x°  +/s2  +  R(R-l)(b2X2V  -a1b1yQz< 


x, 


y2(t2)   =y2   (t2(S6))   -^ 


(2R 


-1)   b2x°     +  2  v/s2  +  R(R-l)(b2x2)2-  a^bjy~2 


and  where   t.  =  t0(S,)   is  given  by 
z    2.      o 


(1)      for     axb  yQ2   >   s2 


t_(S   )   =  — — 
2      6  /aTb 


Jin 


11 


y(t2)  -v/yz(t2)  -  y02  +  s^/Ca^j 
y0  "  s/v^X 


(2)      for   a  b  y  2   <   s2 


W^ 


£n 


11 
(3)      for     a.^^2  =  s2 


V/2(t2)    -  yQ2  +  s2Aa1b1) 

-  y^  ] 

I              s//a1b1   -  yQ 

4 

2    6       v^T 


fcn 


11 


y(t2) 


Case  2.      a  b  y  2  ^  s2  +   (R-l)(b2x°)2 


TQ  <;  T1  <   T    (S?) 

where  T   is  given  below 


T(S?)=   t1(Sy)    + 


^2 


tanh 


-1 


b2x°    Sr~ 


nvr 


/a1b1y02-S2   +   (b2x^)' 


Table  III.   Solution  to  Prescribed 


Duration  Battle  for  0  £  6  <  1. 
(cont.)-5 


S_:   (continued) 


and  where  t-  =  t  (S_)   is  given  by 


t,(S7)   =  -±2 
1      7  /aTb 


In 


11 


^albly02   "  s2  +   (b2X2)ir     "  b2X2 


O   A 


*VT  yo " s 


In  both  the  above  Cases  1.  and  2.  we  have  T  given  by 
(1)   for  a  b  y  2  >  Rs2 


T      =  — — 

0    /rs 


An 


2   2 


rv^^   -  R{s2-(blXo)^}    -  bxx°    /R 


'/a1b1  yQ   -  s/r 


(2)      for     a  b  y   2   <   Rs2 


To~ 


^K 


£n 


bixi  ^~-/a1b1y0z  -  R{s2-(blX°)2> 


sviT  -  /a1b1  yQ 


(3)   for  a^^2  =  Rs2 


T0  = 


^2*2 


in 


b2x° 


Furthermore,  we  must  have 


t3  *  Tx  -  x1 


(i.e.  T1   -  t3  £  t  ) 


where  x  =  x  (S  )   is  given  by 


T,    = 


v/^Ib 


Jin 


2    2 


+  /zz   +  a2   -   1 
1  +  a 


and     z    = 


R-6 
R-l 


a   = 


,     b9 

r  .  /      2 


and  where   t„  =  t_(S_)   is  a  root  of  the  equation 


F(t3)  =  0, 
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Table  III.   Solution  to  Prescribed 

Duration  Battle  for  0^6  <  1. 
(cont.)-  6 


S_:   (continued) 
and  we  have 

F 


(t3)   =        2"(1  +  —  )|s   cosh   jy/a2b2    (Ti  '   t3)   +  ^l^i   t3( 

-  ^YQ   sinhjv^  (Tl   -   t3)   +     ^t3j] 

+  |d    -   ~| ■)[■    COShjv^"    (Tl    -    tj)     -    ^   t3j 

+  /^yQ  sinh  j^  (Ti  -  *3>  -  ^A  ^j] 

-  s  cosh   /a  , •   t_  +  /a1b1  y_  sinh  /a.b   t„  +  b  x„ . 
Solve  the  above  equation  F(t~)=0  by  Newton-Raphson  method. 


(n+l)=    (n) 
3        3 


'(S(n))  ' 


where 
F'(t3)=^(l4) 


s  sinh  {/a  b   (T  -  t  )  +  /a  b   t  } 


-  v/a1b1  yQ  cosh  i^a^  (T±  -   t3)  +  /a.^  t^ 


^A  1 

2         U        R; 


s   sinh   {/a  b2    (T-    -   t-)    -   v/a1b1   t3> 


+   /a1b1  yQ  cosh   { /a2b2    (^   -  t3)    -   /a^   t3> 


-  s    /a  b     sinh      /a  b      t     +  a  b  yfl  cosh   /a  b      t»< 
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Table  III.   Solution  to  Prescribed 

Duration  Battle  for  0  <  6  <  1. 
(cont.)  -  7 

S  :   (concluded) 

There  are  two  cases  to  consider  for  starting  value  for  Newton-Raphson 

method,   t    : 

Case  a.   a^y^  <  s2  +  (R  -  l)(b2x°)2 

t3(o)  =  t2(s6)  -(t(s6)  -tJ. 

where  t_(S,)  and  T(S-)  are  given  above. 
lb  b 

Case  b.   a  by  2  ^  s2  +  (R  _  1)^^)2 

<o) "  W  "  {I(V  -  Ti}> 


C3 


where  t  (S7)  and  T(S_)  and  given  above. 


NOTE:   No  extremal  leads  to  S   if: 

(1)  there  is  no  root  to  the  equation  F(t-)  =  0, 
or 

(2)  we  do  not  have   0  £  t_  £  T  , 
or 


(3)  we  do  not  have 


x1(t3)  >  0, 


x2(T1)  =  0, 


y(\)   >  o. 
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Table  III.   Solution  to  Prescribed 

Duration  Battle  for  0  s  S  <  1. 
(cont.)  -  8 

S4:  Xl(T)  >  0,  x2(T)  >  0,  y(T)  -  0,  T  *  T 

Since  y(T)  =  0,  the  switching  time   T,  =  x  (S, )   is  given  by 


Tl  =  W  =  ~ 

/a_b_  vaTb 


cosh"1  (|=£-)-  -i-  In   (z  +  ^P-rr), 


2  2  2  2 

R-5 


where  z  = 


R-l  ' 


Case  1.   t  ^  T 


Extremal  Control:   4>*(t)  =  0  for  0  £  t  £  T, 

Domain  of  Controllability:   a.b  y  2  £  Rs2<l  -  l/z2l, 

albly02  <  r{s2  -  (b^)2}, 
albly02  <  s2  +  (K-DCV^2' 

i        -i  iSSM  T 

T  =  — — — -  tanh    \ >£  T.  . 

^bj  Is)1 


Note:   for  0  €.   6  <  R  -  /R(R  -  1)  optimal  pathes  also 

satisfy  (equality  yielding  the  dispersal  surface) 
for   0  <.   Xl°  <  b2x°  /(k  b1) 

v   2   d  2   n  (  v   o  [z2(R-l)  +  R]   ,,  o    , 
albiy0  S  RS   "  M  Vl        2R +  b2X2  ] 

where  k  is  given  by  k  =  -jz2  -  R(z-1)2>  /(2R) 
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Table  III.   Solution  to  Prescribed 

Duration  Battle  for   0  <;  5  <  1. 
(cont.)  -  9 


S, :    (continued) 
Case  2.   x   <  T 


Subcase  A.   R-/R(R-1)        1 
Optimal  Control:      <J>*(t)  = 


1     for     0  £  t  ^  T-t,  where   t,    =      xn(S.) 

1  114 


t 0      for      T-t      <    t   ^  T 


is  given  above 


Domain  of  Controllability:   a  b  y  2  >  s2  +  A(b_x_)2 


T=x1  + 


In 


2*VT 


(z 


albly02  <  s2  +  B(b2x2)2 


aibiyo2  >  Rs2  f1  -  i/z2! 

/R(z2-1))  (s  +  /a1b1  y^ 


.  (s  -  /a1b1  y0)  (z  +  /R(zz-1)). 


<.   T. 


Subcase  B.   5  =  R  -  /R(R-l) 
Optimal  Control:      §   *(t)  = 


1   for   0  <.   t  £  t   <  t 


tl(s2) 


0   for   t   <  t  «£  T 


where   tn  =  t-  (S_)  =  

1   1    2        /Tb 


&n 


11 


b2x2 


and  tc   is  any  value  such  that   0  S  t,  <  t., 


Domain  of  Controllability:   a  b-  y  2  =  s2 


V2>vf(^"1) 
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Table  III.   Solution  to  Prescribed 

Duration  Battle  for  0^5  <  1. 
(cont.)-  10 

S.  :   (concluded) 

Note;   In  this  case  the  optimal  control  is  not  unique.   Consequently 

T  is  not  unique,  but  it  lies  within  the  bounds 


I  /a2b2  y0 
/a2b2 


Furthermore,  we  must  have  T  ^  T_ . 


S5:  X]L(T)  =  x1(t1)  =  0,  x2(T)  >  0,  y(T)  =  0,   T  £  T± 

!1  for  0  £  t  £  t     where  x  (t  )  =  0 
0  for  t  <  t  ^  T 

Domain  of  Controllability:    a-b^  2  ^  s2  +  B(b2x  )2 

albly02  <  s2  +  (R"1)(b2x2)2 

T  =  t  +  ~—^7  tanh   )  * 

/a2b2         (       b2x°  /r 

where  t  =  t  (S  )  =  t  (S.)   is  given  by 
(1)  for  a^yQ2  >  s2 


t,    =  — — —     £n 


1  J    VlV     •  S'  +   (b2XP7     ■  b2X2 


|  ft"  j  

\fa!hl  (  ^ 


1-1  yo  "  s 


G-28 


Table  III.   Solution  to  Prescribed 

Duration  Battle  for  0  £  5  <   1. 
(cont.)-ll 


S_:      (concluded) 


(2)      for  a1b1  yQ2  <   s2 


ti  =  T1111      ) / ( 

*??i     (  s  -  ,/vT  yo  ) 

(3)     for  a]_b1  yQ2  =  s2 


1     j~t      boxo 

'a  b1  I    2   2 


NOTE:   for  0  £  6  <  R  -  ^(R-l)   optimal  paths  also 

satisfy  (equality  yielding  the  dispersal  surface) 
for  0  ^  x_  <;  kb^x  /b2 

k2  o2  R       ),      o      [z2(R-l)   +  R]     ,    .       o  )2 

albly0     *     Rs     -  ^    )Vl  2R  +  b2X2  |    ' 

where     k     is   given  by     k  =  {  z2   -  R(z-l)2  (   / (2R) . 
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Table  III.   Solution  to  Prescribed 

Duration  Battle  for  0  £  6   <   1. 
(cont.)-  12 

S6:   Xl(T)  =  x1(t2)  >  0,   x2(T)  =  0,   y(T)  =0,   T  <;  T 

where  t~  <  T 

1  for  0  as  t  as  t. 
Extremal  Control:   <t>*(t)  -  { 

0  for  t«  <  t  35  T 

where  t~  =  t  (Sfi)   is  the  smallest  t  such  that 

alV2(t2»  "  R  {(Vl<*2>  +  b2X2  )2  -  (Vl(t2>)2} 

An  explicit  expression  is  given  for  t_  below. 
Domain  of  Controllability:   a.  by.2  £,  s2  +   A(b„x?)2 

a^  a  R  {s2  -   (blx°)2} 

aiVo2  *  s2  +  ^^'(Vp2 


i      .-i  (RV2 +t/s2  +  R(R-i)(ypz-  w7~  |  _ 

"  cosh        \ —     >  ^  T 

v^b^"  (  (R-Db2x°  +  /s2  +  RCR-DCb^p^-a^^j 


where   t_  ■  t?(S,)   is  given  below  and  y(t_)   used  in  the  computation 


of   t   is  given  by 


— o- 
x. 


y(t2)  =\/  a    J(2R-l)bx°  +  2/s*  +  R(R-l)(b  x")^  -  a^Q2 


2 
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Table  III.   Solution  to  Prescribed 

Duration  Battle  for  0^6  <  1. 
(cont.)-  13 


S, :   (concluded) 
6 

Also,      t2  =   t2(sf-)      is  given  by 
(1)      for     a^yQ2   >   s2 


i     J  y(t2)  -  /y2(t2)  -  yQ2  +  sJ4/(a1b1) 


(2)  for  a^^Q2  <  s2 


,  /yz(t2)  -  yQz  +  s^/Ca^)  -  y(t2) 

to  =  "TZH   ^n 
2   ZaTbT 


11 


(3)  for  a^^Q2  =  s2 


sf/I^l  -  yQ 


C2  = 


^     ^y(t2)f 


NOTE  1:   We  also  have  that  b  x°(R-l)  <.   b  x° 


NOTE  2:   for  0  «£  6  <  R  -  /R(R-l)    optimal  paths  also  satisfy 

(there  is  a  dispersal  surface  (whose  equation  hasn't  been 
explicitly  determined)  in  the  solution) 


0  ^  x2  ^  kb-x  /b2 


where  k  is  given  by  k  =  \z2   -   R(z-l)2}/ (2R) . 


G-31 


Table  III.   Solution  to  Prescribed 

Duration  Battle  for  0  s  5  <  1. 
(concluded)-  14 

Sy:   X;L(T)  =  ^(t.)  =  0,   x2(T)  =  0,   y(T)  >  0,   T  <;  T. 

where  t1  <  T 

{1  for  0  ^  t  5!  t.        where  x  (t  )  =  0 
0  for  t  <  t  <;  T 

Domain  of  Controllability:   a  b  y  2  >  s2  +  (R-l)(b  x°)2 

1  -1  (  b2X2   ^  ) 

T  =   t.    +     -^-     tanh  1{  l   l  U  T 

^?2"  /ab  y   2   _  s2  +   (b  x^)2     J 


where  t1   is  given  by 


t,  =  — — 


/albly02  -  s^  +  (b^)2  -  b2x° 


,/5lEl      <       "SX  y0  "  S 


Sg:  X]L(T)  =  0,  x2(T)  =  x2(t4)  =  0,  y(T)  >  0,   T  <;  Tj 


where  t   <  T 

4 


NO  EXTREMALS  LEAD  TO  THIS  END  STATE 
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S 
For  S,   (which  corresponds  to  C   and  C    of  the   Isbell- 

Marlow  problem)  the  optimal  control  can  be  determined  explicitly 
when  when  there  is  a  switch  in  tactics  for  R  -  /R(R-l)  <  6  <  1. 
This  is  because  the  solution  for  S,  is  essentially  the  same  as  for 
the  corresponding  part  of  the  Isbell-Marlow  problem.   For  the  latter 
problem  (see  Appendix  F)   we  know  that  optimal  paths  lead  to  C,. 
only  for  R  -  /R(R-l)   <  6  <  1. 

In  Case  (2)   the  optimal  policy  cannot  be  expressed  in  the 
very  simple  form  as  in  the  first  case.   When  Y  wins  in  time  less 
than  T   (S   for  which  the  optimal  policy  is  determined)  the  solu- 
tion is  precisely  the  same  as  when   5^1.   However,  for  all  other 

cases  (i.e.  terminal  states  S..  through  S.)  the  extremal  policy  is 

I  b 

to  finish  the  prescribed  duration  battle  by  firing  at  X  ,  regard- 
less of  whether  or  not  X   has  been  annihilated.   This  differs 
from  that  when  6  ^1.   Thus,  we  see  that  the  initial  force  levels 
may  (indirectly)  affect  the  optimal  target  selection  policy. 

Finally,  in  considering  Tables  III  the  following  may  be  use- 
ful: 


(a)   for  0  £  6  <  R  -/R(R-l) , 

we  have  A  >  B  >  0,  (2) 


(b)  for   6  =  R  -/R(R-l), 

we  have  A  =  B  =  0,  (3) 

(c)  for  R  -  /R(R-l)   <  5  <  1, 

we  have  A  <  B  <  0.  (4) 
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The  quantities  A  and  B  appear  in  inequalities  defining  various 

domains  of  controllability  and  are  defined  by  (see  also  Section  3) 

R(z2  -  1)  -  z2 
A  =  A(R,z)  =  • 

(z  -  D2  (5) 

and 

R(z2  -  1)  -  z2 
B  =  B(R,z)  =  * 

z2  (6) 


6.   Development  of  Solution. 

We  will  now  discuss  the  application  of  our  solution  proce- 
dure which  we  outlined  in  Section  4.   Because  many  of  the  devel- 
opments parallel  those  already  given  in  detail  for  the  Isbell- 
Marlow  problem  (see  Appendices  A  and  F) ,  we  will  omit  (or  outline) 
them.   Possibly  we  may  provide  more  details  in  the  future.   Several 
novel  (and  new)  aspects,  however,  are  examined  in  detail  below. 

In  applying  our  solution  algorithm  one  must  first  consider 
the  basic  necessary  conditions  of  optimality.   However,  since  we 
have  done  this  in  detail  in  Section  4.b.(l)  of  Appendix  E  (only 
the  boundry  conditions  of  the  dual  variables  differ  between  the 
prescribed  duration  battle  and  the  terminal  control  case),  we  will 
only  summarize  the  main  results  here  with  the  interested  reader  be- 
ing directed  to  Appendix  E  for  details. 

We  showed  that  in  order  for  it  to  be  optimal  to  have  x. (t)  =  0 
for  a  finite  interval  of  time  we  must  have 

a1b1  ^  a2b2-  (7) 


At  entry  (when  t  =  t  )  to  a  constrained  subarc  on  which  x  =  0, 
we  have  (from  the  corner  conditions) 


PiCtr> =  ^  p2  cv). 


(8) 


p.  (t,-)  -  P.(t.,+)  for  i  = 


2,3, 


(9) 


where  t  -  denotes  a  left-hand  limit. 

Additionally,  we  showed  that  when  x  >  0  and  x     >   0  the 
extremal  control  (determined  by  the  maximum  principle)  is  given  by 


<J)*(t)  = 


(1     for  v(t)  >  0, 


0  for  v(t)  <  0, 


(10) 


where 


v(t)  -  a.(-p1  (t))  -a_(-p0(t))  -  a,p„(t)  -a.p^t).  (11) 


lv  ^1 


'2V  r2 


2r2 


'lrl 


Furthermore,  v(t)  4   0  almost  everywhere.   Differentiation  of  (11) 
and  combination  with  the  adjoint  equations  yields 


of  =  "P3(t)  (albl  "  a2b2} 


(12) 


Since  it  is  readily  shown  that  p~(t)  >  0  for  t  <  T,  we  have  that 


dv 

-j-  <   0  for  all  t  <  T, 

at 


(13) 


where  our  nonrestrictive  assumption  a  b..  >  a  b_   should  be  re- 
called. 
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Considering  the  above,  the  time  history  of  the  extremal 
control  is  readily  developed  for  each  terminal  state  once  the 
boundry  conditions  on  the  dual  variables  have  been  determined.   For 
S  ,  S  ,  and  S   the  length  of  the  battle  is  equal  to  T  ,  and  the 
boundry  conditions  of  the  dual  variables  are  shown  in  Table  IV. 
It  is  to  be  noted  that 

for   S  ,  S  ,  and  S~ 

p3  (t  =  T)  =  r  >  0  (14) 

For   S.  ,  S_,  and  S.   the  length  of  the  battle  is  determined  by  the 
4    5       o 

time  to  annihilate  Y  so  that  y(t  =  T)  =  0  is  an  equality  con- 
straint on  a  state  variable  at  an  unspecified  terminal  time.   The 
boundry  conditions  of  the  dual  variables  are  shown  in  Table  V.   It 
is  to  be  noted  that 


for   S ,  ,  Sj.,  and  S, 


p3(t  =  T)  =  0.  (15) 

Moreover,  for   S„ ,  and  S,   we  have  that 
J       b 

-1-  <    -p  ft  =  T)  £  q.  (16) 

a2         2 

The  synthesis  of  extremal  control  and  determination  of  the 
domains  of  controllability  are  similar  to  developments  in  Appen- 
dices A  and  F.   Consequently  they  are  omitted  here.   There  are 
two  interesting  aspects  that  we  encountered  in  doing  this  work 


Table  IV.   Boundry  Conditions  of  Dual  Variables 
for  Terminal  States  S..  ,  S  ,  and  S_. 


For  S1>  S2>  and  S  : 


px(t  =  T)  =  -p  +  \>v 
where 


vi 


=  0  for  x  (T)  >  0, 


2>  0  for  x1(T)  =  0, 


p2(t  =  T)  =  -q  +  v2, 


where 

=  0  for  x. (T)  >  0, 


v 
2 


;>  0  for  x  (T)  =  0, 


P3(t  =  T)  =  r. 
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Table  V.   Boundry  Conditions  of  Dual  Variables 
for  Terminal  States  S,,  S  ,   and  S,, 


For  S . ,  Sc,   and  S , : 
4   j         b 


Px(t  =  T)  =  -p  +  Vl, 
where 


v 
1 


=  0  for  x1(T)  >  0, 


^  0  for  x, (T)  =  0, 


'1 


p2(t  =  T)  =  -q  +  v2, 
where 


V2 


=  0  for  x2(T)  >  0, 


^  0  for  x,(T)  =  0, 


p  (t  =  T)  =v~,  where  v3   is  unrestricted 


However,  the  transversality  condition 
H(t  =  T,  x±,  p±,  <j>*(t  =  T))=  0 
yields  that 

P3(t  =  T)  =  0. 


(most  of  the  details  of  which  we  have  omitted) .   These  are 

(a)  when  a  switch  in  the  target  type  upon  which  all  Y-fire  is 
concentrated  occurs  without  the  annihilation  of  a  target 
type,  the  switching  time  depends  upon  the  initial  force 
levels  and  possibly  the  valuation  of  Y  survivors. 

(b)  when  P  =  (x  ,  x_  ,  y  )  is  such  that  when   5  <  1  an  extremal 

g 
leads  to   S.    (i.e.  we  reach   S.   with  a  switch  in  tactics) 
4  4 

with  T(S,  )  <  T  ,  we  can  also  possibly  steer  the  system  to 
an  end  point  with  y(T  =  T  )  =  0  without  violating  any 
necessary  conditions  of  optimality. 

7.   Dependence  of  Switching  Time  on  Force  Levels  and  Valuation  of 
Y  Survivors. 

We  consider  the  case  when   5  <  1.   Then  we  saw  for  the  Isbell- 
Marlow  problem  (see  Appendices  A  and  F)  that  an  extremal  policy  might 
be  to  shift  the  concentration  of  all  Y-fire  from  X   to  X   before 
the  annihilation  of   X  . 

Now,  there  are  two  subcases  for  entry  to   S,.   We  consider 

the  extremals  for  which 

!1   for   0  ^  t  £  T  -  x 
0   for  T  - ■  t-  <  t  S  T.  (17) 

Then,  let  us  say  that  an  extremal  leads  to   S.   when  we  use  the 
extremal  control  (17)  and  reach  the  terminal  state  x  (T)  >  0, 
x  (T)  >  0,  y  (T)  =  0,  T  £  T  .   We  denote  the  domain  of  controll- 
ability for   S,   when  (17)  is  used  by  D(S  )  and  the  corresponding 
payoff  by  P..   It  is  convenient  to  introduce  the  "backwards  time" 


G-39 


t  defined  by   t  =  T-t.   Then  according  to  previous  arguments  (see 
Appendix  A  where  ^(S^   is  the  same  as   x  (C  )) ,  we  have 

T1(V         cosh"1  z,  (18) 


where 


R-5  ,   s 

z  =  r=T  '  <19> 


Now  for  S1:  x±(T)    >  0,  y(T)  >  0,  T=T  , 


we  have 


P3(t=T)  =  r  (20) 

Also,  recalling  (11),  we  have 


v(t=0)  =  alP-a2q,  (21) 

so  that  v(t=0)  <  0  when  6  =  a  p/(a  q)  <  1.   Recalling  (13),  which 

may  be  written  as  —  >  0,  we  see  that  as   t   increases  v(x) 

dx 

must  change  in  sign  from  negative  to  positive.   Let  us  denote 
this  backwards  time  by   t  .   Then  by  (11) ,  we  see  that 
<J>*(T)  =  0  for  0  ^  x  ^  t..  .   We  may  use  this  extremal  control 
to  integrate  the  adjoint  equations  backwards  in  time.   We  may 
use  the  results  to  obtain 


v(t-)    =      (a1b1"a2b2^ 


£—  cosh  /a„b„    x  +  -  sinh  /a_b0    t 

2  2   2  ^~2  2   2 


+a  p  - 


aibiq 


**  b2 


(22) 
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The  "backwards  time"  of  the  first  switch  in  tactics,  denoted  by 
t   is  determined  by  v(t=t  )  =  0,  and  this  readily  yields 


where 


1  x      ^K 


In 


2  +  /zz  +  az   -  1  ), 
1  +  a       '  I 


(23) 


a  = 


(24) 


We  next  prove  a  useful  lemma, 


LEMMA  1:   Assume  that   5  <  1, 
Then 

<   0  , 


3a 


(25) 


where   t  =  t..  (S  ) 
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PROOF : 

A.   We  readily  compute  from   (23) 


3x 


1   a  +  1  -  z2-z  /z2+a2-l 


3a   '  /^b~  (l+a)»/z2-h>2-l  (Z+  /z^a^-T  )  (26) 


B.   Now   5  >  1   implies  that   z  >  1,  so  that 
a  <  za  <  z/z2-faz-l   , 
whence    a  +  1  -  z2  -  z  /z2+a2-l  <  0, 
which  proves  the  lemma.  Q.E.D. 

Now  by  Lemma  1  we  readily  see  that 


_A  =  1-  12.  <  o  ,  (27) 

3r     3a   3r 


sxnce 


la  =  i-v  Z-2-    o  .   Furthermore,  it  is  easily  shown  that 
3r   q  V  a2 

lim  t1(S1)  =  0.  <28) 

r-++°° 


Thus  for   S  ,  if  we  value  the  survival  of  our  own  forces  high 
enough,  we  never  fire  at  X,,  when  x^t)  >  0  even  though 
a  p   <  a  q.  (To  be  precise,  when   5  <  1,  we  can  make  the  time 


interval   [T  -  t  ,  T]   during  which  we  concentrate  all  fire  on 
X~  arbitrarily  small  be  taking  r   large  enough.) 

Let  us  now  show  that  when  the  optimal  policy  is  to  change 
the  allocation  of  fire  (all  concentrated  on  one  target  type)  with- 
out a  target-type  force  level  being  driven  to  zero,  we  fire  at  X 
longer  when  y(T)  =  0   than  when  y(T)  >  0   (assuming  that  R  >  1 
and   6  <  1).   This  is  the  significance  of  the  theorem  that  we  now 
state  and  prove  as  Theorem  1. 

THEOREM  1:   Assume  that  R  >  1  and   6  <  1. 
Then, 

Ti(si)  <  w- 

PROOF : 

A.   Let  us  observe  that  both  (18)  and  (23)  may  be  con- 
sidered to  be  special  cases  of  the  general  equation 


T,     = 


•S5! 


in 


+  /zz  +  az   -  1 
1  +  a 


(29) 


where 


a  =  SlJLJl  J -A 

q        V   a„ 


(30) 


We  note   that 


(a)      for     y(T)    >   0      (i.e.      for   S^ , 
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v  =  0, 

(b)  for  y(T)  =  0  and  T  <  T    (i.e.  for  S  ), 

v  =  -r 
B.   Clearly,  we  have  by  (23)  and  (29) 


^1  m*i   3a 

3v    3a  3v 


(31) 


C.   Observing  that  -r-^  =  r-^  f^  and  f*-  =  — \/— 

3v    3a   3v     3v   q  V  a 

>  0,  the  theorem  readily  follows  by  (31)  and  Lemma  1,  since 

-  r  =  v(S^)  <  v  (S  )  =  0.  Q.E.D. 

Thus,  we  see  that  y(T)   affects  the  length  of  time  that 
Y  fires  at  X   when  there  is  a  change  in  the  optimal  tactic 
(concentration  of  all  fire)  before  annihilation  of  X  .   Intuitive- 
ly, we  see  that  firing  longer  at  X   prolongs  the  length  of  battle 
for  those  cases  when  y(T)  =  0,   since  a-,b,  >  aob?.   We  will  prove 
this  precisely  in  the  next  section  and  show  that  this  is  not  an  opti- 
mal tactic. 


8.   The  Case  When  T  =  T.,  for  S, 

1       4. 


In  the  previous  section  we  conjectured  that  when  R  >  1  and 
5  <  1  ,   we  could  prolong  the  length  of  battle  leading  to   S,   by 
firing  longer  at  X- .   Let  us  consider  the  following.   Assume  that 
we  have  P°  =  (x°,  x°,  yfi)  given  and   6  <  1.   Further  assuming  that 
P°   is  such  that  an  extremal  leads  to  S   with  T(S^)  <  T^  we  re- 
call that  p~(t=T)  =0.    We  will  show  that  by  adjusting  the  value 


of  p  (t=T)  =  y  >  0,  we  can  steer  the  system  to  y(T)  =  0  and 
T  =  T  (i.e.  we  can  prolong  the  length  of  battle  by  the  valuation 
placed  on  our  forces).   Finally,  we  will  show  that  this  can  never 
be  an  optimal  policy  in  order  to  drop  this  special  case  from 
further  consideration  (as  we  have  done  in  Tables  III) . 

We  again  consider  the  extremal  control  (17)  which  leads 
to   S,  :   x  (T)  >  0,  x  (T)  >  0,  y(T)  =  0,  T  <;  T  .   In  the  special 
case  when  T  =  T- ,   the  transversality  condition  no  longer  holds 
so  that  p  (t=T)  =  v   where   v   is  unrestricted.   Let  us  write 
v^  =  r  +  v   so  that 

*;  -  tiw  -  7=  *-(z + /zi :  f  - 1  )■         ^ 

/a2b2 


a  .  ^ff   •  (33) 

Using  the  extremal  control  (17),  we  readily  obtain  from  the  state 
equations  (1) 

y(T-x1)  =  yQ  cosh  /I^iT-x^ —  sinh  /a^  (T-^)  ,     (34) 

b^a-T^   +   b2x°  =  s  cosh  /a1b1  (T-^)  -   A^  YQ  sinh  /a^CT-i^) ,   (35) 

and 

, (b.x  (T-r  )  +  b  x°)       

y(T)  =  y(T-T  )  cosh  •albT  t — M-  sinh  /ajb^  t^.     (36) 
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(34),  (35),  and  (36)  are  readily  combined  to  yield 

■^jp*  =  "  (b1x1(T-x1)  +  b2x°)  cosh  Slfo   Tl 

+   R  /a2b2  y(T-T1)  sinh  /a^bj  x±.  (37) 

Now  T  is  defined  by 

y(T)  =  0.  (38) 

Combining  (36)  and  (38)  with  (37),  we  obtain 

3v(T)     . . l   (blX  (T-t  )  +  b  x°)2  I 

*gp  .  R^  y(T-Tl)  sinh  ^  ,tf-         l^yi   (I,Ty    [.  (39) 

Let  us  recall  the  "generalized   square  law" 

for     <J>(t)   =  constant     for  all     t  €  [t.  ,t_] 

?2(t=tx)   -  C2(t=t2)  =  |'J>a1b1  +  (l-<j,)a2b2}  ly^Cfe-fej.)   -  y2(t=t2)j,      (40) 

where  s(t)  =  b  x  (t)  +bx2(t).   Now  (17)  and  (40)  yield 

s2  -  a^^2  =  (b1x1(T-x1)  +  b2x°)2  -  a1b1y2(T-x1).   (41) 

Thus,  (39)  and  (41)  yield  the  following  lemma 

LEMMA  2:   Consider  y(T)   given  by  (36)  and  T 
defined  by  y(T)  =  0.   Then 

9y5T)  <  0  if  and  only  if 
3T 

aiVo2  K  s2' 


It  is  convenient  to  obtain  the  following  from  a  backwards 
integration  of  the  state  equation  of  the  state  equations  (1) 
using  (17) 


b2X2  =  b2x2(T)w  +  b1x1(T-T1)(w-l) 


(42) 


We  may  then  combine  (36) ,  (38)  ,  (17),  and  (40)  to  obtain 


y^Or-O  = 


(w2-l)(s2-aib  yQ2) 


V     '   a1b1  -  w^a^-a^)    * 


(43) 


and  further  analysis  yields 


Vs2   -  a  b  y  2 
R  -  w4-l) 


(44) 


and 


s2   -  a,b.y  2 
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b2x2(T)    =  b2x2      +    (1-")Vr-w2(R-1)         ' 


(45) 


where 


w  =  w(a)   =   cosh  N/a2b      t    (a)    . 


Now,  we  may  use  (43)  to  solve  (34)  for  T  -  t   to  obtain 


(46) 


T  -  t  = 
1 


2/1^ 


in 


(w  -  /R(w2-1)) (s  +  /a1b1  yQ) 
(s  -  /a1b1  yQ)(w+  /Rtw2-!)) 


(47) 
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Combining  (32)  and  (46),  we  obtain 


z  -  a  /z2  +  a2  -T   - 
; 2 for   0  £  a  <  1, 

X  ™*  o 


z2  ■ 


w(a)  =   < for  a  =  1, 

2z 


d  +  a2-  1  -  z  for  a  >  1.      (48) 


a' 


Differentiation  of  (48)  yields 


3w    -(zct  -  /z2  +  a2  -  l)2       -    n 
— — —  =  — i <£ —      for   0  ^  a  <  1 

/z2  +  a2  -  1   (1-a2)2      and  a  >  1,         (49) 


and  hence 

3w 
8a 


<  0  for  0  <;  a  <  1  and  a  >  1.  (50) 


Considering  (47),  it  is  easily  seen  that  for  P  €  D(S.)  with 
T  =  T   and  a  b.y  2  <  s2,  we  must  have 


1  s;  w  <WR  -  1  .  (51) 

Since  T  is  implicitly  determined  by  (38) ,  which  we 
re-write  as 

y(ir  T)  -  0,  (52) 

partial  differentiation  readily  leads  to 


(53) 


3y     <      3x1 

3T 

9t             3v 

3v 

3v_ 
3T 

Now, 


Ha 

3v 


3T. 


3a   3T1   3a 


3a   3v 


3a   3v 


,  or  using  (33) 


3v        q  V  a2  3a 


(54) 


where  we  recall  Lemma  1  (see  (25)). 

Now,  partial  differentiation  of  (36)  and  use  of  (34)  and 
(35) (including  partial  differentiation  of  (34)  and  (35)  with 
respect  to  t  )  yields 


3y(T) 


±-1-  -  -  (R-l)/a2b2  y  (T-x1)  sinh  v^bj  t^ 


so  that 


^     <  0   for  R  >  1. 
9T1 


We  now  state 

THEOREM  2:     Assume  that   R  >  1,  5  <  1, 


aiVo2  <s2,andUw<\   R- 


Then 


il    >  0. 

3v 


(55) 


(56) 


(56) 


PROOF:   Immediate  by  (51),  (53),  (54),  (56),  and  Lemma  2.   Q.E.D 


Thus,  we  see  that  by  increasing  the  implicit  valuation  of  Y- 
forces  (i.e.  v)  the  length  of  battle  can  be  extended  until  T  =  T,  , 
To  sum  up  for  S , :  x  (T)  >  0,  x„(T)  >  0,  y(T)  =  0,  T  ^  T  ,  in  the 
special  case  when  T  =  T       we  have  so  far  found  no  violation  of 
any  necessary  condition  of  optimality  in  having  -r  £  v.   Then 
there  is  a  region  in  the  initial  state  (force  level)  space  from 
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which  extremals  lead  to  this  special  end  condition,  and  in  this 
region  extremals  also  lead  to  S,  with  T  <  T  (i.e.  recall  that 
for  such  P°  T  <  Tx  when  we  take  p  (t-T)=0). 

We  now  show,  however,  that  although  the  length  of  battle 
can  be  extended  so  that  y(T=T  )=  0,  this  results  in  a  reduced 
payoff.   Recalling  that  we  denote  the  payoff  associated  with  the 
extremal  control  (17)  by  P.,  we  can  use  (44)  and  (45)  to  obtain 


P     =-* 
4       b2 


/r"1\k     °         /R  -  wz(R-l)V/S      "  alVo     (  ,_ 

-Z(ir)b2X2      -{ R ;Vr-w^(R-1)      (     '  (57) 


Differentiation  of  (57)  and  use  of  (33)  readily  yields 

9P.    (R-l)   3w  (s2-a.biyn2)1/2 
3V    ^   9a   (R-w2(R-l))3/2 

Considering  (48)  and  (50) ,  we  have 

w(a=0)  =  z  and  ~  <  0  for  0  £  a  <  1  and  a  >  1.  (59) 

Hence, 

w(a)  <  z  for  0  <  a  .  (60) 

Thus,  (50),  (58),  and  (60)  yield 

THEOREM  3:   Assume  that  R  >  1,  5  <  1, 


aiVo2  K   *2'  and  x  *w  ^^T    ' 


Then      ._ 
4 
9v 


9.   Discussion. 

There  are  two  topics  that  especially  merit  discussion: 
(1)  the  structure  of  the  optimal  target  selection  policy  and  (2) 
the  development  of  solutions  to  such  problems.   We  also  suggest 
a  future  research  task. 

From  our  work  reported  here  in  Appendix  G  on  the  prescribed 
duration  battle,  we  conclude  that  the  structure  of  the  optimal 
target  selection  policy  may  depend  (indirectly)  on  (initial)  force 
levels.   This  revises  a  tentative  conclusion  stated  in  our  earlier 
work  [3]  (see  also  Appendix  B) .   To  elaborate  further,  if  no  target 
type  is  annihilated,  then  there  is  no  explicit  dependence  of  the 
optimal  target  selection  policy  on  force  levels.   However,  the  non- 
annihilation  of  all  target  types  itself  depends  upon  the  initial 
force  levels.   Furthermore,  we  saw  that  for   6  =  a  p/(a  q)  <  1 
the  optimal  length  of  time  that  Y   fires  at  X   depends  on 
whether  or  not  y(T)  =  0.   Moreover,  for   5^1.  not  only  does  the 
optimal  target  selection  policy  take  an  extremely  simple  form 
(concentrate  all  fire  on  X   while  x  >  0)  but  it  also  is  inde- 
pendent of  the  force  levels.   When  5  <  1,  the  solution  was  ex- 
tremely complex  except  for  the  case  when  Y  annihilates  both 
X   and  X   in  which  case  the  same  simple  optimal  policy  as  when 
6^1  applies. 

It  seems  appropriate  to  compare  the  structure  of  the  optimal 
target  selection  policy  for  the  prescribed  duration  battle  with 
that  for  the  terminal  control  (f ight-to-the-f inish)  battle.   We 
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have  seen  that  we  may  consider  the  fight  to  the  finish  to  be  im- 
bedded in  the  prescribed  duration  battle.   For  such  cases  when 
T  <  Eg,  the  solutions  of  the  two  problems  are  identical.   Further- 
more, when   6^1   the  structures  of  the  optimal  policies  were 
identical.  We  conjecture  that  this  will  always  be  true  for 
n-versus-one  combat  when  surviving  target  types  are  valued  in 
direct  proportion  to  their  kill  capabilities.   However,  if  the 
initial  force  levels  are  such  that  the  specified  maximum  duration 
for  the  battle  (denoted  by  T  )  terminates  the  battle,  then  the 
structures  were  different  when   6  <  1.   Thus,  we  see  that  even  the 
nature  of  the  scenario  for  a  target  selection  problem  may  have  an 
effect  upon  the  optimal  policy. 

In  developing  a  (hopefully)  complete  solution  for  the  pre- 
scribed duration  battle,  we  have  encountered  some  novel  and  inter- 
esting features  that  require  further  discussion.   First  of  all,, 
it  does  not  seem  possible  to  develop  an  entirely  analytic  solution 
to  such  problems  (see  Tables  III).  Computational  methods  must  be 
combined  with  analytic  results  arising  from  theoretical  optimal- 
ity  conditions  in  order  to  determine  the  optimal  policy.   It 
should  be  noted  that  difficulties  (multiple  extremals  from  an 
initial  point  P  )   arise  in  the  n-versus-one  battle  when  target 
types  are  not  valued  in  direct  proportion  to  their  kill  rates. 
In  the  second  place,  the  theory  of  state  variable  inequality 
constraints  (SVIC)  must  be  used  to  solve  such  problems.   We  have 


already  pointed  this  out  at  several  places  elsewhere  in  this  re- 
port (see  Appendixes  E  and  F).   Thirdly,  in  all  the  problems  that 
we  have  studied  in  the  same  depth  as  for  the  problem  at  hand,  in 
developing  a  complete  solution  we  have  invariably  come  across 
some  special  cases  that  require  an  inordinate  amount  of  analysis. 

For  this  prescribed  duration  battle,  one  such  case  was  for  S. 

4 

when  T  =  T  .   However,  such  special  cases  require  treatment  in 
order  that  a  complete  solution  be  obtained. 

Let  us  point  out  that  for  the  simplest  (deterministic) 
optimal  control  problems  of  target  selection  that  we  have  now 
studied  a  tremendous  amount  of  analysis  has  been  required  to 
develop  a  complete  solution.   We  do  not  believe  that  any  other 
type  of  model  or  formulation  (for  example,  discrete-time  formula- 
tions solved  by  nonlinear  programming  or  the  formalism  of  dynamic 
programming)  would  reduce  any  of  the  key  difficulties  which  have 
required  the  majority  of  the  analysis.   We  have,  for  example,  con- 
sidered a  discrete-time  version  of  the  Isbell-Marlow  problem  (see 
Appendices  A  and  F)  and  used  nonlinear  programming  (Kuhn-Tucker 
conditions)  to  develop  the  basic  optimality  conditions.   Again, 
for  the  finite-dimensional  problem,  there  is  no  way  to  circumvent 
the  non-uniqueness  of  extremal  paths  leading  to  different  terminal 
states.   Similar  remarks  apply  when  one  uses  the  formalism  of  dy- 
namic programming. 
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One  task  remains  to  be  done,  and  we  propose  this  to  ONR 
as  a  future  research  task:   examination  of  the  sensitivity  of  the 
payoff  to  the  target  selection  policy.  With  our  solutions  to 
such  problems  (i.e.  both  terminal  control  and  prescribed  duration) 
available,  it  would  seem  desirable  to  examine  whether  or  not  a 
simple  (probably  non-optimal)  target  selection  policy  might  yield 
nearly  the  same  return  as  the  true  optimal  policy.   For  example, 
for  the  prescribed  duration  battle  we  have  seen  that  the  solution 
is  amazingly  complex  when  6  <  1.   In  fact,  we  can't  explicitly 
state  the  optimal  policy  in  general  but  can  only  numerically  com- 
pute the  optimal  policy  for  a  given  set  of  input  values.   A  simple 
target  selection  policy  which  would  yield  close  to  the  optimal 
return  would  clearly  be  of  great  value  in  this  case. 
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Appendix  H.   On  the  Solution  to  Lanchester-Type  Equations  of  Modern 
Warfare  with  Variable  Coefficients. 

1.   Introduction. 

This  appendix  develops  solutions  to  extensions  of  F.  W. 
Lanchester's  classical  equations  of  modern  warfare  (frequently 
referred  to  as  aimed-fire  equations)  for  combat  between  two  homog- 
eneous forces.   In  these  extensions  the  lethality  of  the  fire  (as 
expressed  by  the  Lanchester  attrition-rate  coefficient)  depends 
upon  time.   When  the  dependence  is  arbitrary,  the  solution  is  an 
infinite  series  of  recursively  related  integrals;  in  special 
cases,  more  convenient  representations  (including  representation 
in  terms  of  tabulated  functions)  are  available.   Solutions  are 
obtained  in  the  following  cases:   (1)  lethality  of  each  side's 
fire  proportional  to  a  power  of  time  and  both  lethalities  initially 
zero,  and  (2)  lethality  of  each  side's  fire  linear  with  time  but 
only  one  side's  lethality  initially  zero.   The  latter  case  models 
the  constant  speed  approach  between  forces  whose  weapons  have 
different  maximum  effective  ranges. 

In  a  previous  note  published  in  the  open  literature  [33]  we 
developed  a  solution  to  variable-coefficient  Lanchester-type 
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equations  of  modern  warfare  (various  Lanchester-type  equations  of 
warfare  are  reviewed  and  our  terminology  explained  below)  for  combat 
between  two  homogeneous  forces  in  the  special  case  of  a  constant 
ratio  of  Lanchester  attrition-rate  coefficients.   A  sketch  of  some 
previous  work  on  variable-coefficient  Lanchester-type  equations  is 
to  be  found  there.   In  the  present  paper  we  develop  solutions  to 
extensions  of  F.  W.  Lanchester' s  classical  equations  of  modern 
warfare  in  which  the  lethality  (or  effectiveness)  of  fire  (as 
expressed  by  the  Lanchester  attrition-rate  coefficient)  may  change 
over  time.   Our  results  point  out  the  complexities  of  analytically 
treating  such  extensions  of  Lanchester' s  classical  work  (see  [24]). 

Our  present  work  considers  only  the  purely  formal,  mathematical 
aspects  of  the  Lanchester  theory  of  combat.   We  hope  that  others  will 
continue  to  address  the  equally  important  scientific  questions  as  to 
whether  such  hypotheses  as  considered  here  are  tenable  in  the  light 
of  empirical  evidence  (see,  for  example,  [17],  [18],  [34],  [35],  [37]). 
We  must  also  note  that  S.  Bonder's  excellent  work  [4],  [6]  on  the 
Lanchester  attrition-rate  coefficient  (see  also  [2],  [22])  very 
naturally  leads  to  the  requirement  of  having  solutions  to  variable- 
coefficient  Lanchester-type  equations  available  to  analysts  in  force- 
structuring  and  weapons  system  design  studies.   The  results  of  this 
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appendix  are  hopefully  a  step  in  that  direction.   Other  recent  papers 
have  discussed  the  estimation  of  parameters  for  the  Lanchester  attri- 
tion-rate coefficient  [30],  [31]. 

Similar  approaches  towards  variable-coefficient  Lanchester- 
type  equations  have  been  previously  used  by  T.  Oberbeck  [28]  and 
F.  Dashiell  and  W.  Fain  [9].   A  summary  of  Oberbeck's  work  and  other 
early  work  on  Lanchester-type  equations  is  to  be  found  in  the  summary 
report  by  R.  Snow  [32].   Dashiell  and  Fain  [9]  base  their  work  on  a 
theoretical  paper  by  Whyburn  [36],  but  do  not  cite  Oberbeck's  pioneering 
work  or  any  of  the  numerous  excellent  applied  mathematical  treatments 
of  systems  of  first  order  ordinary  differential  equations  (see,  for 
example,  pp.  53-55  of  [15]  or  pp.  73-77  of  [23]).  Although  Whyburn's 
method  of  successive  approximations  is  applicable  to  systems  of 
variable  coefficient  differential  equations,  Dashiell  and  Fain  consider 
only  the  constant  coefficient  case.   This  work  is  the  mathematical  basis 
of  later  tactical  simulation  studies  using  matrix  Lanchester-type 
equations  [7],  [11]. 

In  this  appendix  we  first  review  various  Lanchester-type  equations 
of  warfare.  Then  we  show  a  simpler  derivation  of  our  previous  result 
for  a  constant  ratio  of  attrition-rate  coefficients.   This  development 
is  later  used  for  insight  into  cases  when  an  infinite  series  may  be 
represented  by  tabulated  functions.   Next,  we  develop  a  general  solu- 
tion to  variable-coefficient  Lanchester-type  equations  of  modern 
warfare  for  combat  between  two  homogeneous  forces  in  the  form  of  an 
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infinite  series  of  recursively  related  terms  by  considering  an 
equivalent  integral  equation.  We  then  consider  some  special  types 
of  attrition-rate  coefficients  for  which  this  series  reduces  to  tabu- 
lated functions  and  an  important  case  for  which  it  does  not.   Several 
numerical  examples  are  given.   Finally,  we  discuss  the  significance, 
applications, and  extensions  of  our  results. 

2.   Some  Lanchester-Type  Equations  of  Warfare. 

It  seems  appropriate  to  briefly  review  some  Lanchester-type 
equations  that  have  appeared  in  the  literature.   In  our  discussion 
here,  we  will  consider  equations  with  attrition-rate  coefficients 
that  depend  upon  time,  although  in  the  original  sources  cited  these 
were  assumed  to  be  constant. 

In  1914  in  an  article  published  in  the  British  journal 
Engineering  (see  [24]),  F.  W.  Lanchester  hypothesized  that  combat 
between  two  opposing  forces  "under  modern  conditions"  could  be 
modelled  by  (see  [34]  for  a  discussion  of  the  assumptions  inherent  in 
(1)  with  constant  lethality  of  fire) 

dx      ,  x 

dT=  "a(t)y' 

^r=-b(t)x,  (1) 

where  x(t)   and  y(t)   refer  to  the  numbers  of  X-  and  Y-forces, 

respectively.  The  functions  a(t)   and  b(t)   are  usually  termed 

[2],  [6],  [22]  the  Lanchester  attrition-rate  coefficients  and  represent 
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the  lethality  of  each  side's  fire.  We  shall  refer  to  equations  (1) 
as  Lanchester's  equations  of  modern  warfare.  These  equations  are 
appropriate  when  both  sides  use  aimed  fire  and  target  acquisition 
times  are  negligible  [8],  [34]. 

This  paper  concerns  the  solution  of  equations  (1).   However, 
other  differential  equation  combat  models  may  be  referred  to  as  Lan- 
chester-type  equations.  Although  not  explicitly  given  by  Lanchester 
[24],  the  following  equations  were  hypothesized  to  describe  hand-to- 
hand  combat  (one  against  one)  in  "ancient  warfare" 

£=-b(t).  (2) 

Lanchester  did  give  [24]  the  following  equations  for  area  fire  (see 
[34]  for  discussion  of  assumptions) 

—  =  -  a(t)xy, 

^=-b(t)xy.  (3) 

Equations  (3)  are  also  appropriate  for  aimed  fire  when  target  acquisi- 
tion time  is  much  larger  than  the  time  to  destroy  an  acquired  target 
and  target  acquisition  time  is  inversely  proportional  to  target  density 
[8].   More  recently,  S.  Deitchman  [10]  used  the  following  equations  to 
model  ambush  situations  (in  guerrilla  warfare)  in  which  the  X-forces 
(ambushers)  use  aimed  fire  against  ambushees  who  return  area  fire 
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■ft-   "  a(t)xy, 

^r=-b(t)x.  (A) 

The  a's  and  b's   (Lanchester  attrition-rate  coefficients) 
may  be,  of  course,  different  in  equations  (1)  through  (4)  and  related 
to  different  physical  quantities.  Other  factors  such  as  replacements, 
operational  losses,  effects  of  supporting  fires,  etc.,  may  be  accounted 
for  by  appropriate  modifications  of  the  above  equations,  but  we  will 
not  consider  these. 

The  above  differential  equation  models  of  combat  are  determi- 
nistic, always  yielding  the  same  result  for  given  initial  conditions. 
However,  such  attrition  equations  are  commonly  assumed  to  represent 
the  mean  course  of  battle  (implying  an  underlying  probability  distri- 
bution [32]).   Other  than  remarking  that  this  is  why  we  will  refer  to 
x(t)   as  the  average  X-force  level  (this  is  well-known  to  be  only 
an  approximation  [32]),  we  will  not  discuss  stochastic  aspects  further. 

3.   Constant  Ratio  of  Attrition-Rate  Coefficients:   Alternate 
Development  1. 

We  derive  a  previously  obtained  result  [33]  in  such  a  manner 
as  to  shed  light  on  the  circumstances  under  which  our  general  result 
of  the  next  section  reduces  to  tabulated  functions.   Let  us  consider 
equations  (1) .   We  have  previously  shown  that  it  is  inessential  whether 
we  consider  time  or  force  separation  (range)  as  the  independent  variable 
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(see  [33]  for  a  transformation  to  change  a  solution  with  time  as  the 
independent  variable  to  one  with  range  as  the  independent  variable) . 
We  may  combine  the  two  equations  of  (1)  into  the  following 
single  equation  for  x(t) 


d  f  1    dx") 

dtla(t)   dt/ 


(t)x  =  0, 


(5) 


with  initial  conditions 


x(t=0)  =  x   and 


dx 


a(t)   dt 


=  -  y 


t=0 

(X. 


0  * 


Introducing  a  new  independent  variable  u  = 
form  equation  (5)  into 


a(s)ds,   we  may  trans- 


d2x   b(t) 


du' 


a(t) 


x  =  0  . 


(6) 


rt 


a(s)ds   to  exist  in  order  for 


0 


We  note  that  we  require  the  integral 
this  transformation  to  be  applicable.   At  this  point,  the  form  of 
equation  (6)  makes  our  previous  result  obvious.   It  also  is  helpful  in 
obtaining  solutions  and  approximations  in  other  cases.   Thus  when 
b(t)/a(t)  =  constant   or  a(t)  =  k  h(t)   and  b(t)  =  k,h(t),   we  have 

d2x   \ 


du" 


x  =  0, 


with 


x(u=0)  =  xQ, 


du"  (u=0)  =  "  V 


so  that 


x(u)  =  x   cosh/k/k   u  -  y  /k  /k  sinh/k  /k   u. 
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Hence 


x(t)   =  xQ  cosh  6(t)   -  yQ^~J\  sinh  9(t),  (7) 

ft 

h(s)ds   and  a  similar  expression  may  be  obtained 
0 


where   8(t)  =  A.  k 
for  y(t). 

There  is,  however,  a  much  greater  significance  of  equation  (6). 
What  we  really  have  is 

j^+   I(u)x  =  0,  (8) 


ft 


where   I(u)  =  -b(t)/a(t)   and   u  = 


a(s)ds . 
0 
Equation  (8)  is  the  "normal  form"  of  equation  (5)  (after  Felix  Klein, 

see  p.  158  of  [13]).   There  is  an  infinite  series  solution  in  terms 
of  repeated  integrations  (first  developed  by  Lord  Kelvin,  see  p.  120 
of  [14])  available  for  equation  (8)  (also  see  p.  351  of  [29]).   We 
shall  not  pursue  this  approach  further  here,  however. 

The  term  I(u)   is  called  the  "invariant"  of  the  normal  form. 
Richards  (pp.  347-350  of  [29])  has  developed  a  table  of  various  func- 
tions, I(u) ,  for  second-order  linear  differential  equations  written  in 

the  normal  form.   He  claims  that  over  180  of  the  first  220  second-order 
linear  equations  listed  by  Kamke  [21]  can  immediately  be  solved  by  use 
of  this  table.   Kamke 's  [21]  classic  work  contains  among  other  things 
a  tabulation  of  solutions  to  second-order  linear  differential  equations, 
similar  to  a  table  of  integrals.   This  means  that  if   I(u)   does  not 
take  a  form  contained  in  Richards'  tables,  then  it  is  extremely  unlikely 
that  equation  (8)  has  a  solution  expressible  by  any  of  the  standard 
functions  of  analysis  and  that  some  type  of  infinite  series  is  the  best 
that  we  can  hope  for. 
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4.  Arbitrary  Attrition-Rate  Coefficients:   Integral  Equation 
Approach. 

In  this  section  we  derive  a  general  solution  to  equations  (1) 

by  means  of  an  infinite  series  of  terms  with  repeated  integrations. 

Our  approach  is,  of  course,  formally  equivalent  to  approaches  mentioned 

earlier  (see  pp.  53-56  of  [15]  or  pp.  73-77  of  [23]). 

By  repeated  integrations,  we  may  obtain  the  following  Volterra 

integral  equation  from  equations  (1) 


rt 


x(t)  =  xQ  -  yQ 


rt 


a(s1)ds1  + 


?1 


a(s1)ds1 


b(s2)x(s2)ds2.     (9) 


Equation  (9)  is  readily  solved  by  Picard's  method  of  successive  approxi- 
mations (see  p.  109  of  [12]  or  pp.  61-74  of  [16])  to  yield 


rt 


x(t)  =  xQf(t)  -  yQ 


a(s)g(s)ds, 


(10) 


where 


f(t)  «  I      H  (t),    g(t)  =  I      K  (t), 
n=0  n=0 


(ID 


and 


for  n  >  0 


HQ(t)  =  1,   KQ(t)  =  1, 


H  (t)  = 
n 


K  (t)  = 
n 


rt 

0 

rt 


a(s ,)ds1 


b(Sl)dSl 


b(s2)Hn_1(s2)ds2, 


a(s2)Kn_1(s2)ds2.  (12) 
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The  formal  steps  above  are  justified,  for  example,  when  a(t)   and  b(t) 

are  bounded  for   te[0,T],   since   f(t)   is  then  majorized  by   cosh/AB  t 

where  A  =  max  a(t) ,   etc.   Under  these  circumstances  it  is  easy  to 

te[0,T] 
show  the  convergence  to  this  method  of  successive  approximations  (see 

pp.  113-114  of  p-2]  or  pp.  61-64  of  [16]),   and  that   f(t)   is  an 
absolutely  and  uniformly  convergent  series  for   te[0,T}. 

Equations  (10) ,  (11)  and  (12)  are  a  general  solution  to 
the  Lanchester-type  equations  (1).   Being  general,  this  solution  is 
complex  but  still  is  in  a  form  amenable  to  computer-aided  computation. 
An  infinite  series  solution,  however,  provides  little  insight  into  the 
effect  on  battle  outcome  of  attrition  rate  parameters,  short  of  our 
generating  parametric  curves  by  grinding  out  solutions  for  specific 
parameter  values.   A  computer  may  not  always  be  available  to  aid  in 
the  summing  of  the  infinite  series.   Hence,  it  is  desirable  to  know 
when  the  series  solution  (10)  may  be  expressed  in  terms  of  tabulated 
functions.   In  the  next  two  sections  we  examine  two  cases  of  when  this 
is  possible.   We  omit  verification  that  (10)  reduces  to  the  well-known 
constant  coefficient  solution  when  a(t)   and  b(t)   are  constants, 
since  this  has  already  been  done  by  Dashiell  and  Fain  (see  pp.  94-95 
of  [9]),  whose  approach  may  be  shown  to  be  equivalent  to  that  given 
here. 
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5.   Constant  Ratio  of  Attrition-Rate  Coefficients;   Alternate 
Development  2. 

We  show  that  (10)  reduces  to  our  previous  result  (7)  when  a(t)  = 
k  h(t)   and  b(t)  =  lch(t).   This  present  derivation  plus  our  first 
alternate  are  useful  in  providing  insight  into  the  circumstances  under 
which  (1)  possesses  exponential  solutions. 

00 

We  develop  the  first  term,   f(t)  =  £  H  (t) ,   in  equation  (10). 


n=0 


n 


From  our  assumed  conditions,  we  have  that 


H 


1      aTb   Jn 


0 


h(s1)ds1 


0 


h(s2)ds2, 


which  by  a  differentiation  and  subsequent  integration  may  be  shown  to 
be  equal  to 


Hx(t)  =  Sp- 


h(s)ds 


Using  the  fact  that 


rs. 


h(sx) 


h(s2)ds2 


n 


(n+1)  ds. 


\< 


s2)ds2 


(n+1) 


}, 


we  can  easily  establish  inductively  that 


<kA> 
Hn(t)  "  "(fnTT 


n  r- 


rt 


h(s)ds 


2n 


(13) 
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ft 

h(s)ds},   where  we  have  made  use  of  the 

0 

power  series  expansion  for   cosh  x.   A  similar  expression  is  readily 


so  that  f(t)  =  cosh{/k~ k~" 


obtained  for   g(t) ,   and  thus  (7)  follows  from  (10)  ,  (11)  and  (12)  in 
this  special  case.   However,  we  now  see  that  the  simple  "exponential" 
solution  (7)  depends  on  H  (t)   being  given  by  (13)  .   This  apparently 
will  occur  only  when  a(t)/b(t)  =  constant.   Thus,  this  is  the  only 
instance  that  we  have  encountered  so  far  in  which  variable-coefficient 
Lanchester-type  equations  of  modern  warfare  have  such  a  simple  solution. 

6.   Another  Example:   Power  Attrition-Rate  Coefficients. 

A  case  of  probably  greater  practical  interest  is  when  the  attrition- 
rate  coefficients  in  (1)  take  the  following  form 

a(t)  =  k  tm, 

a. 

n 


b(t)  =  l^t  ,  (14) 


where  we  will  refer  to  m  and  n  as  the  exponents  of  the  power 
attrition-rate  coefficients.   We  shall  sketch  the  development  of  the 
solution  via  the  Volterra  integral  equation  as  given  by  (10)  ,  (11)  and 
(12). 

We  focus  on  the  development  of   f(t)  in  (10).   Combining  (12) 
and  (14) ,  we  find  that 


Hl(t)  "  Vb  |   Sldsl 


s,         ,  ,   m+n+2 
m 


k  K  t 


s„ds„  = 


2   2    (n+1) (m+n+2) ' 
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where  we  must  place  the  following  restrictions  on  m  and  n:  m  >  -1 
and  n  >  -1.   It  is  readily  established  by  mathematical  induction  that 


\(t> 


(kkb)K  k(m+n+2) 


k  k_1 
k !  (m+n+2 )   n   {  n+l+i  (m+n+2  )  } 

1=0 


Equation  (15)  may  be  rearranged  to  yield 


(15) 


lyt)  =  r(i-p) 


\h 

2s 

2k     2ks 


k!T(k+l-p)  ■ 


and  hence, 


f(t)  =  I    h.  (t)  =  r(i-p) 
k=o  * 


'■*3P 


2s 


tpS  I 


-P 


f/k~k~ 
a  b   s 
t 


(16) 


(17) 


where 


s  =  (m+n+2) /2,     p  =  (m+1)/ (m+n+2) , 


and   I  (x)   is  the  modified  Bessel  function  of  the  first  kind  and  order 
v 

v.   It  has  the  following  power  series  expansion 

v+2n 


Mx>  ■     I 
n=0 


© 


^  n!r(n+l+v)  ' 


A  similar  result  is  to  be  obtained  for  g(t).   Thus,  we  obtain  the 
following  expression  for  the  average  strength  of  the  X-forces 


m+1 


x(t)  =  t 


x0r(i-P) 


f/k   kip 
a  b 


2s 


r/k 


-p 


a  d   s 


kay0r(i+p) 


f/k 


(m+1) 


A 
2s 


{A 


at)   s 


(18) 
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Since  few  Bessel  functions  of  fractional  order  are  tabulated,  we  also 
observe  that  this  solution  (18)  is  also  equivalent  to  the  following 
power  series 


x(t)  = 


x0rd-p) 


W(14?) 

(nrt-1) 


2k 


2ks 


k!r(k+l-p) 


2k   2(ks+ps) 
k!T(k+l+p) 


(19) 


Similar  expressions  are  obtainable  for  y(t).   The  same  results  can  be 
obtained  by  considering  (8)  and  using  Richards'  table  (see  p.  349  of 
[29]).   In  this  particular  instance,  probably  the  easiest  way  to  obtain 
(18)  and  (19)  is  to  observe  the  ordinary  differential  equations  satis- 
fied by  Bessel  functions  (see  pp.  166-167  in  [19]). 

When  m  =  n,   then   s  =  m  +  1   and  p  =  1/2   in  (18).   Observing 

that 

/~2 

I1/0(x)  =/ —  sinh  x,   etc.   (see  p.  165  of  [19]) 


■1/2 


and  that 


r(i/2)  =  fi  , 


we  can  easily  show  that  (18)  reduces  to  (7)  in  this  special  case. 

It  is  of  interest  to  the  operations  research  worker  who  desires 
numerical  solutions  that  Lebedev  (see  p.  112  of  [25])  notes  that  J. 
Liouville  has  shown  that  the  case  of  half-integral  order,  i.e., 
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v  ■  n  +  1/2  where  n  is  an  integer,  is  the  only  case  where  Bessel 
functions  reduce  to  "elementary  functions."  Hence,  except  for  the  case 
m  =  n  it  is  not  possible  to  express  x(t)   as  given  by  equation  (18) 
in  terms  of  elementary  functions. 

7.   Weapon  Systems  with  Different  Effective  Ranges;   Linear  Attrition- 

Rate  Coefficients. 



Consideration  of  the  attrition-rate  coefficients  given  by 

equations  (14)  leads  us  to  realize  that  they  apply  only  to  combat 

between  forces  using  weapons  with  the  same  effective  range,  i.e., 

attrition  commencing  for  both  sides  at  the  same  time.   Consider  the 

example  of  a  constant  speed  attack  of  a  mobile  force  against  a  static 

defense.   Assuming  that  firing  by  either  side  commences  when  its  enemy 

is  within  the  effective  range  of  the  weapon  system,  we  see  that  (14) 

has  implicit  within  it  the  assumption  that  both  forces  achieve  nonzero 

attrition  rates  at  the  same  range.   Of  interest  is  the  case  where 

opposing  weapon  systems  have  different  effective  ranges.   An  example 

of  time  dependent  attrition-rate  coefficients  which  correspond  to  the 

lethality  of  each  side's  fire  depending  linearly  upon  range  in  such  a 

scenario  is  given  by 

a(t)  =  k  t, 
a 

b(t)  =  k^t+A).  (20) 

The  range  dependence  of  these  Lanchester  attrition-rate  coefficients 
is  depicted  in  Figure  1. 
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We  become  aware  of  the  great  generality  of  the  solution  as  given 
by  (10) ,  (11)  and  (12)  when  the  attrition-rate  coefficients  in  equations 
(1)  are  given  by  (20) .   In  this  case,  we  have  not  been  able  to  develop 
a  solution  in  terms  of  tabulated  functions  via  Richards'  tables  [29] 
after  (1)  has  been  converted  to  the  normal  form  (8).   However,  the 
following  solution  is  readily  established  inductively 


•w-ojXH  -  y°  Jk  (I  G-(t)i 


(21) 


where 


rA 


Fn(t)    = 


"*-!. 


and  for     n   >   0,     we  have 

1 


M 


2n        3n       n 


(2n>!  ki0    a 


v     „k,k  n-k 


for     k  =  0, 


B     = 
n 


and 


4n(4n-2) 


_(4n-k)  (4n-k-2)_ 


i     (4n-2)      n-1 
3      (3n-2)      n-1 


k  „k-l 

B      .    +  B 
n-1  n-1 


G    (t) 
n 


^S 


(2 


for      1  -£  kin  -  1, 


for     k  =  n, 


2n+l        3n+2        n 


k.k  n-k 


t y     cVt 

2n+l)!    ,*n     n     t 


k=0 


V1' 


and   for     n   >   0 ,      we  have 

1 

4n(4n+2) 


_(4n-k)(4n-k+2)_ 

4      (4n+2)      n-1 
3      (3n+2)      n-1 


k  „k-l 

C      .    +  C 
n-1  n-1 


for     k  =   0, 


for     1  £  k  a  n  -  1» 


for     k  =  n, 
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A  similar  result  is  readily  obtained  for  y(t).   It  is  easily  seen  that 
(21)  reduces  to  our  previous  results  when  A  ■  0. 

The  form  in  which  we  have  written  this  solution  (21)  is  of  parti- 
cular value  for  numerical  computations,  since  it  is  of  the  form 
(for  t  >  0) 

x(t)  =x„f(x1,x0)  ■  y0/j~  g(x1,x2)  , 


ko  v  i'  r     '0i 


\ 


where  X.   and  X~  are  parameters  defined  by 


X   -± 
A2   t  * 

This  allows  us  to  develop  tabulations  of   f   and   g,   depending  on 

the  two  parameters   X   and   X  ,   which  may  be  used  to  generate 

numerical  solutions  to  all  such  problems  as  we  have  considered  in 

this  section.   These  functions  are  similar  to   cosh  X    and   sinh  X 

(depending  on  a  single  parameter),  to  which  they  reduce  when  A  =  0, 

i.e. ,   X„  =  0. 

It  is  of  interest  to  note  that  if  the  infinite  series  method  of 

Frobenius  (see  pp.  78-97  of  [13]  or  pp.  132-139  of  [19])  is  used  to  solve 

the  above  problem,  we  have  not  been  able  to  obtain  the  solution  in  the 

form  of  (21) ,  although  the  series  so  obtained  is  readily  verified  to  be 

a  rearrangement  of  (21) .   The  absolute  convergence  of  the  series  in 

question  justifies  this  rearrangement.   This  shows  that  the  solution 
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representation  given  by  (10) ,  (11)  and  (12)  is  a  particularly 
important  one,  since  it  leads  to  results  of  practical  value  that  we 
have  not  been  able  to  obtain  by  any  other  means. 

8.   Some  Numerical  Examples. 

In  this  section  we  examine  two  numerical  examples.   S.  Bonder 
[3],  [5]  has  pioneered  in  the  study  of  range  capabilities  and  mobility 
considerations  for  weapon  systems  in  combat  described  by  Lanchester- 
type  equations  of  modern  warfare.   Our  numerical  results  illustrate 
how  his  type  of  analysis  may  be  extended  to  weapon  systems  with  (1) 
different  range  dependencies  of  lethality  of  each  side's  fire  (but  the 
same  effective  range)  and  (2)  linear  attrition-rate  coefficients  but 
different  effective  ranges. 

We  consider  a  constant  speed  attack  of  a  defended  position  with 
the  combat  described  by 

dx      ,    N       /n    r  .m 

dT=  _a(r)y  =  _a0(1  "  R~}  y' 

a 

jZ   =  _3(r)x  =  _e  (1  _JL}n  x>  (22) 

3 

where  R   and  R   are  effective  ranges  of  the  Y-  and  X-force  weapon 
a        (3 

systems,  i.e.   a(r)  =  0   for   r  >  R  .   Range  is  related  to  time  by  the 


relationship 


r(t)  =  RQ  -  vt  , 


where  R   is  the  opening  range  of  battle.   Several  range  dependencies 
for  attrition-rate  coefficients  are  illustrated  in  Figure  2. 
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Numerical  results  (after  an  example  of  S.  Bonder,  see  Tab  H, 
p.  IV-34  of  [5])  are  shown  in  Figures  3,  4,  5  and  6.  These  curves  have 
been  generated  by  a  computer  program  using  the  series  solutions  (19) 
and  (21),  as  appropriate.   To  check  the  computer  algorithm,  the  series 
solution  (19)  was  compared  with  the  solution  given  by  (18)  in  terms  of 
modified  Bessel  functions  of  the  first  kind  and  fractional  order  (it 
is  easily  seen  that   0  <  p  <  1) .   Available  tables  of  modified  Bessel 
functions  are  discussed  in  [1]  (see  pp.  455-456)  and  [26]  (see  p.  41). 
We  made  use  of  the  National  Bureau  of  Standards'  tables  [27]  in  our  work, 
Although  tables  for   I   only  exist  for   v  =  +  1/4,  +  1/3,  +  2/3  and 
+  3/4   (see  also  p.  235  of  [20]),  interpolation  methods  may  be  used  for 
other  fractional  values.   In  [27]  Lagrange  interpolation  polynomial 
(8th  degree)  coefficients  are  given  which  yield  four  decimal  place 

accuracy  for   I  (x)   for   0  <  v  <  1.   The  above  considerations  are  use- 

J  v 

ful  when  one  does  not  have  a  computer  readily  available  for  computation. 

In  Figures  3  and  4  both  systems  have  the  same  effective  range, 

i.e.,   R  =  Rn,   and  the  battle  commences  at  this  range,  i.e.,   R  =  R  . 
ot     p  u     ex 

In  this  plot  we  have  held   a  =  a(r=0)   and   3   constant  and  have  var- 
ied the  exponents,   m  and  n,   of  the  range  dependencies  of   a(r)   and 
3(r),   respectively.   S.  Bonder's  pioneering  work  [3],  [5]  has  exten- 
sively studied  the  cases  of  m  =  n.   We  see  that  the  nature  of  the 
combat  trajectory  is  quite  sensitive  to  the  particular  combination  of 
ra  and  n,   and  that  these  are  additional  parameters  which  help  deter- 
mine who  wins  or  loses.   (In  the  special  case  when  m  =  n,   a  square 
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R   =  R    =  R       =  2000  meters 
o      a       £ 

a  =0.06    X-cas./(min.  x  Y-unit) 
o 

/3  =  0.6       Y-cas./lmin.  x   X-unit) 
v  =  5  m.  p.  h. 

V  ,0»  V  30 


15 

Average 

force     10  -- 
strength 

x(t) 


500  1000  1500 

Force  separation,  a  (meters) 


2000 


1*0    denotes  m=l  ,  n  =0. 

The  symbol  x  denotes  the  end  of  a  force- 
level  trajectory  due  to  the  annihilation  of 
enemy  forces. 

Figure  3.  Force- level  trajectories  of   X  for 
various  combinations  of  exponents, 
m  and  n,  in  power  attrition- rate 
coefficients. 
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RQ=  RQ  =   Rq  =  2000  meters 

a    =  0.06   X-cas./(min.  x  Y-unit) 
o 

B  -  0.6      Y-cas./(min.   x    X-unit) 
Ho 

v    =  5  m.  p.  h. 

x    =    10,  y  =  30 
o  Jo 


30-- 


Average 

force 
strength    2  0 

y(t  ) 


10 


500  1000  1500 

Force  separation,    <*,    (meters) 


2000 


l:3    denotes  m  =  I  ,  n  =3. 

*-* 

The  symbol  x  denotes  the  end  of  a  force - 

level  trajectory  due  to  the  annihilation  of 
enemy  forces. 

Figure  4.    Force- level  trajectories  of  Y  for 

various  combinations  of  exponents, 
m  and  n,in  power  attrition-rate 

coefficients. 
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F*    =  R    =  2000  meters 
o        a 

aQ=  0.06    X-cas./(min.  x  Y-unit) 
@o=   0.6       Y-cas./ (min.  x    X-unit) 
v    =   5  m.  p.  h. 
xo  =    ,0^o=  30 
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The  symbol  x  denotes  the  end  of  a  force-level 
trajectory  due  to  the  annihilation  of  enemy 
forces. 

Figure  5.  Force -level  trajectories  of    X  for 
various  effective  ranges  of    X- 


force  weapons,  Rn 
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RQ=  RQ=  2000  meters 
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/3  =0.6  Y-cas./(min.  x   X-unit) 
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Figure  6.    Force-level  trajectories  of  Y  for  various 
effective  ranges  of    X- force  weapons, 
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law  which  is  independent  of  these  parameters  relates  the  X-  and 
Y-force  strengths.)   If  we  refer  back  to  Figure  2,  it  should  be  clear 
why  this  is  so:  m  and  n  reflect  how  weapon  system  kill  rates 
increase  with  closing  range  to  their  peak  values  at  "point  blank"  range, 
If  we  consider  the  Y-force  to  be  the  attacker  who  employs  a  weapon 
whose  capability  varies  linearly  with  range,  i.e.,  -m  =  1,   then  the 
battle  may  have  quite  different  outcomes  depending  on  the  value  of   n. 
The  reader  should  contrast  the  battle  trajectories  denoted  as   1:0, 
1:1,   1:2,   and   1:3  in  Figures  3  and  4. 

We  also  see  that  we  can  use  the  initial  trend  of  battle  to  fore- 
cast battle  overcome  only  when  we  know  the  nature  of  the  dependence  of 
both  weapon  system  capabilities  upon  range.  Again,  reference  to 
Figures  3  and  4  should  make  this  clear.   For  example,  compare  the  out- 
comes for  curves  denoted  as   1:2,   2:2,   and   3:2.  We  also  should  note 
the  "compounding"  effect  over  time:   a  small  advantage  in  range  capa- 
bility may  materially  effect  battle  outcome. 

In  Figures  5  and  6  we  show  the  effect  of  increasing  the  effective 

range  of  the  defender's  weapons.   Again  we  may  consider  the  X-force 

to  be  the  defender.   For  these  computations  we  have  held  the  opening 

range  constant  at  Rn  =  R  =  2000  meters.   (It  is  a  straightforward 

matter  to  extend  this  analysis  to  the  case  when  R  <  R_  £  R_.)   Both 

a    0    p 

attrition-rate  coefficients  vary  linearly  with  range,   a   and   3~ 

have  been  held  constant,  and  R   has  been  varied.   Except  for  the  case 

B 

in  which  R  =  2000  meters  when  the  solution  to  (22)  is  given  by  (7), 
p 
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we  know  of  no  other  instance  in  which  there  is  a  relationship  between 
x(t)   and  y(t)   that  is  independent  of  time  (such  as  the  classical 
Lanchester  square  law) .   The  curves  in  Figures  5  and  6  exhibit  the 
(obvious)  qualitative  result  of  increasing  the  long  range  capability 
of  the  defender's  weapon  system:  more  attacker  casualties  occur 
earlier  in  the  battle,  and  these  are  then  magnified  by  the  "compounding 
nature"  of  the  Lanchester-type  equations  (22).   However,  by  considering 
such  an  attack  scenario  and  using  analysis  such  as  the  above,  one  might 
quantitatively  assess  the  value  of  the  long-range  firepower  capability 
of  a  weapon  system  (here  the  defender's  weapons),  giving  consideration 
to  the  (hypothesized)  dynamics  of  combat. 

9 .   Comments. 

In  this  section  we  discuss  the  significance,  applications,  and 
extensions  of  the  results  of  this  paper.   Our  research  is  complementary 
in  nature  to  that  on  the  Lanchester  attrition-rate  coefficient  [2],  [3], 
[6],  [22],  [30],  [31].   To  make  use  of  our  work  one  needs  to  know  the 
range  dependence  of  weapon  system  kill  rates  for  the  scenario  under 
study.   The  demonstrated  sensitivity  of  combat  outcome  to  the  type  of 
range  variation  of  weapon  system  kill  rates  makes  important  the  deter- 
mination of  such  range  dependencies. 

Our  results  extend  the  analytic  capabilities  for  studying  combat 
dynamics  in  those  instances  where  time  or  range  dependence  is  signifi- 
cant.  Previous  analytic  results  [3],  [33]  were  limited  to  cases  of 
weapon  system  performances  described  by  kill  rates  with  (1)  the  same 
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type  of  range  dependence  and  (2)  the  same  effective  range.   Our  results 
allow  both  these  restrictions  to  be  removed.   We  may  also  view  previous 
results  as  applying  to  the  instance  when  the  kill  rate  of  one  weapon 
system  strictly  dominates  that  of  the  enemy's  (consider  the  plots  of 
a(t)  =  k  t   and  b(t)  =  kt   when  m  =  n) .   We  have  presented  analysis 
results  which  apply  when  one  weapon  system  does  not  dominate  another 
at  all  ranges.   Such  a  situation  is  shown  in  Figure  7. 

We  should  also  note  that  previous  analytic  results  were  developed 
only  for  when  a  square  law  related  opposing  force  average  strengths. 
In  this  instance  one  could  view  the  effect  of  range  dependent  attrition 
rates  as  transforming  the  time  (range)  scale  of  the  constant  coefficient 
square  law  attrition  process  [33].   Our  more  general  results  apply  even 
in  those  cases  when  (to  the  best  of  our  knowledge)  there  is  no  time- 
dependent  relationship  between  opposing  force  strengths. 

Our  results  may  be  used  to  study  the  effects  of  mobility  and 
weapon  system  range  capabilities  on  combat  dynamics.   One  can  use  our 
results  to  extend  S.  Bonder's  analysis  [5]  of  the  implications 
and  mobility  considerations  of  Lanchester  combat  between  forces  employ- 
ing weapon  systems  with  appreciable  range  variations  of  kill  rates. 
As  noted  above  our  results  apply  under  less  restrictive  circumstances: 
opposing  weapon  systems'  kill  rates  don't  have  to  vary  with  range  in 
the  same  manner,  and  both  weapon  systems  don't  necessarily  have  to  have 
the  same  effective  range. 

Other  extensions  using  the  general  solution  (10)  are  possible  for 
weapon  systems  that  have  different  effective  ranges.  In  this  paper  we 
have  considered  the  illustrative  case  in  which  both  weapon  systems  have 
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Figure  7.    Example  of  weapon  system  not  dominated  at  all 
ranges. 
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kill  rates  which  very  linearly  with  range.   Extension  to  other  range 
dependence  combinations  of  attrition-rate  coefficients  is  straight- 
forward (but  messy) . 

10.   Summary. 

We  have  developed  a  general  solution  to  variable-coefficient 
Lanchester-type  equations  of  modern  warfare  for  combat  between  two 
homogeneous  forces.   This  general  result  was  applied  to  two  specific 
types  of  attrition-rate  coefficients  to  yield  new  analytic  results  for 
Lanchester  combat.   In  general,  series  solutions  are  obtained  by  our 
approach,  but  we  have  discussed  the  circumstances  under  which  these 
may  be  expressed  in  terms  of  tabulated  functions. 

Solutions  now  exist  for  a  wide  class  of  variable-coefficient 
Lanchester-type  equations  of  modern  warfare  as  a  result  of  this  paper. 
We  have  extended  analysis  capabilities  to  include  the  following 
important  cases:   (1)   opposing  weapon  systems  whose  kill  rates  have 
different  range  dependencies  and   (2)   weapon  systems  with  different 
effective  ranges. 
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Appendix  I.   Optimal  Control  of  the  Lanchester 
Stochastic  Process. 

1 .   Introduction. 

In  this  appendix  we  present  preliminary  results  from  our  study 
of  the  optimal  control  of  the  Lanchester  stochastic  process.   Our 
goal  is  to  determine  whether  the  structure  of  the  optimal  allocation 
policy  for  the  selection  of  target  type  at  which  to  fire  is  affected 
by  whether  the  attrition  process  is  modelled  as  being  deterministic 
or  stochastic.   Except  for  the  pioneering  work  of  Isbell  and  Marlow 
[27}  (who  optimized  a  truncated  stochastic  process  and  thus  did  not 
account  for  the  true  dynamics  of  combat) ,  we  are  not  aware  of  any 
other  similar  work  on  target  selection  in  the  Lanchester  theory  of 
combat  when  attrition  is  a  stochastic  process  (although,  of  course, 
numerous  papers  have  appeared  (see,  for  example  [12],  [33])  concern- 
ing the  application  of  optimal  stochastic  control  theory  to  various 
other  different  tactical  allocation  problems).   In  our  initial  inves- 
tigation here  we  shall  consider  a  one-sided  optimization  problem,  with 
our  ultimate  goal  being  the  study  of  the  corresponding  stochastic 
game  (i.e.  two-sided  decision  problem). 

It  should  be  recalled  that  two  basic  approaches  to  analytic 
war  gaming  are   (1)  the  Lanchester-type  deterministic  differential 
equation  models  [5],  [6],  [16]  or   (2)  stochastic  models  (such  as 
DYNTACS  [3],  a  high-resolution  simulation).   In  our  past  ONR  sponsored 
research  we  have  extensively  studied  the  optimal  control  of  the 
deterministic  Lanchester  attrition  process  [38],  [39],  [40],  [41], 


1-2 


[42],  [43].   In  such  optimization  problems  the  differential  equation 
models  of  combat  are  deterministic,  always  yielding  the  same  result 
for  given  initial  conditions.   However,  such  attrition  equations 
are  commonly  assumed  to  represent  the  mean  course  of  battle  (imply- 
ing an  underlying  probability  distribution  [37]).   In  fact,  one  fre- 
quently refers  to   x(t)   as  the  average  X-force  level  (this  is  well- 
known  to  be  only  an  approximation  [37])  and  the  model  as  an  expected- 
value  model. 

As  regards  the  optimal  selection  of  the  target  type  at  which  to 
fire,  it  is  not  clear  whether  such  expected-value  methodology  would 
be  appropriate  to  a  situation  with  small  numbers  of  combatants  and 
a  stochastic  attrition  mechanism  (such  as  occurs  with  DYNTACS  [3]). 
We  base  this  opinion  upon  our  consideration  of  the  work  of  G.  Clark, 
who  studied  (see  Chapter  11  of  [3]  and  [9])  differences  between  deter- 
ministic and  stochastic  homogeneous-force  Lanchester  formulations. 
He  concludes  that  (see  p.  11-19  of  [3])  "the  deterministic  model 
would  have  difficulty  approximating  a  stochastic  simulation"  with 
respect  to  the  time  history  of  force  levels.   Based  on  our  considera- 
tion of  the  above  results  of  G.  Clark,  we  felt  that  it  would  be  appro- 
priate to  investigate  whether  or  not  there  are  significant  differences 
between  optimal  allocation  policies  for  deterministic  and  stochastic 
attrition  processes.   If  such  a  difference  were  to  be  observed,  then 
this  would  raise  the  more  basic  question  of  whether  or  not  different 
study  results  could  be  obtained  merely  by  the  choice  of  modelling 
technique,  i.e.  Lanchester-type  expected  value  equations  or  Monte 
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Carlo  simulation. 

Our  research  approach  is  to  consider  the  same  scenario  (prescribed 
duration  battle  between  a  homogeneous  Y-force  and  a  heterogeneous 
enemy,  X  and  X_)  and  then  compare  the  solution  of  the  deterministic 
model  with  that  of  the  corresponding  stochastic  one.   To  this  end, 
we  have  developed  a  complete  solution  (in  the  sense  that  numberical 
values  may  be  readily  obtained  by  means  of  a  digital  computer  program) 
to  the  two-versus-one  prescribed  duration  battle  (see  Appendix  G)  . 
In  this  appendix  we  consider  the  corresponding  stochastic  control 
problem.   We  will  develop  the  basic  equations  from  which  the  optimal 
target  selection  policy  may  be  determined.   When  these  equations  are 
solved,  results  may  be  compared  with  those  for  the  corresponding 
deterministic  process.   A  M.S.  thesis  student  (R.  Powers,  LT  USN) 
is  currently  carrying  out  the  details  of  the  comparison  of  solutions. 
We  hope  to  make  a  detailed  comparison  in  our  future  ONR  research. 

It  seems  appropriate  to  briefly  sketch  some  background  on  the 
theory  of  stochastic  optimal  control,  which  we  will  apply  to  our 
target  selection  problem.   Bellman's  principle  of  optimality  (see 
p.  83  of  [1])  and  the  concept  of  an  optimal  expected-value  function 
(depending  only  upon  the  current  state  of  the  system)  may  be  thought 
of  as  forming  the  basis  for  optimal  stochastic  control.   (As  pointed 
out  by  Wendell  Fleming  [20]  it  is  a  matter  of  taste  how  broadly  one 
interprets  the  phrase  optimal  stochastic  control;  we  accordingly 
avoid  any  definition  here.)   As  pertains  the  problem  at  hand,  it 
may  be  considered  to  be  a  special  case  of  a  general  model  discussed 
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by  Kushner  [30]  (who  cites  similar  work  in  England  by  Florentine 
[21]).   S.  Dreyfus  has  discussed  the  differences  between  some  types 
of  optimal  control  of  stochastic  systems  [13]  (see  also  [14])  (espe- 
cially the  difference  between  open-loop  and  closed-loop  control, 
which  are  well-known  to  be  equivalent  for  deterministic  systems  [13], 
[25]). 

Recently,  H.  Kushner  has  written  an  introductory  text  on  sto- 
chastic control  [31],  which  contains  many  references  to  the  literature 
(to  which  he  has  been  a  prolific  contributor) .   Other  survey  papers 
(the  first  of  which  contains  an  extensive  bibliography)  are  by  W. 
Fleming  [20]  and  P.  Whittle  [44]. 

2.   The  Lanchester  Stochastic  Process. 

In  1914  in  the  British  journal  Engineering  F.  W.  Lanchester 
[32]  postulated  that  combat  under  the  conditions  of  "modern  warfare" 
between  two  homogeneous  forces  could  be  described  by  the  equations 

dx 

%  =  -  bx,  (1) 

where  a,  b  are  commonly  referred  to  as  the  Lanchester  attrition-rate 
coefficients  and  x(t),  y(t)  are  force  levels.   During  World  War  II, 
B.  Koopman  suggested  a  reformulation  of  such  a  model  in  stochastic 
form  [34].   Subsequent  work  has  been  by  R.  Snow  [37],  R.  Brown  [7], 
[8],  and  G.  Clark  [9].   B.  Koopman  has  called  the  corresponding 
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stochastic  process,  the  Lanchester  stochastic  process  [29]. 

Before  considering  the  control  problem,  it  seems  appropriate 
for  us  to  review  a  few  results  for  the  Lanchester  stochastic  process. 
Consider  combat  between  a  homogeneous  X-force  and  a  homogeneous 
Y-force.   We  consider  a  stationary,  Markov  process  (see  any  standard 
text  [2],  [11],  [35]  for  a  further  discussion).   Let  the  state  proba- 
bility be  denoted  by   P(m,n,t),   where   m   is  the  number  of   x  com- 
batants alive  and  n   is  the  number  of  Y   combatants  alive  at  time 
t.   Thus, 


P(m,n,t)  =  Prob 


there  are  m  X  and  n  Y   survivors 
at  time   t   after  battle  begins 


Making  standard  assumptions  (see  [4]),  we  find  that  the  state  proba- 
bility satisfies  the  following  system  of  differential-difference 
equations 

for   1  ^  m  £  M  and  1  ^  n  ^  N   (also,  we  adopt  the  conven- 
tion that   P(m,n,t)  =  0   for  m  >  M  or  n  >  N) 


dP 

l.m,n,t;  =  t 

-   {A(m,n)  +  B(m,n)}  P(m,n,t),  (2) 


i   -(m,n,t)  =  P(m+l,n,t)  A(m+l,n)  +  P(m,n+l,t)  B(m,n=l) 


where 

M(N)   is  the  number  of   X(Y)   combatants  at  the 
beginning  of  battle  at   t  =  0, 

A(m,n)   is  the  rate  of  attrition  of   X   forces;   A(0,n)  -  0, 
and   B(m,n)   is  the  rate  of  attrition  of  Y   forces;   B(m,0)  "  0, 
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i.e. 


Prob 


one  X  casualty  in  interval 
from   t   to   t+At 


=  A(m,n)At 


(Moreover,  we  see  that   P(m,n,t)   is,  more  precisely,  the  transition 
probability 


P(m,n,t)  =  P(m,n,t;M,N,t=0) 


=  Prob 


m  X's   and   M  X's   and 

n  Y's   at   t  N  Y's   at   t=0 


Of  course,  the  state  space  is  discrete,  i.e.   m  =  0,1..., M 
and  n  =  0,1..., N.   At  the  boundary  of  the  system,  i.e.   m  =  0 
or  n  =  0,   equation   (2)  takes  the  form 

dP 

— (m,0,t)  =  P(m+l,0,t)A(m+l,0)  +  P (m,i,t)B(m,l) 

-  P(m,0,t)A(m,0), 


dP 
dt 


(0,n,t)  =  P(0,n+l,t)B(0,n+l)  +  P (l,n, t)A(l,n) 

-  P(0,n,t)B(0,n), 


dP 
dt 


(0,0, t)  =  P(l,0,t)A(l50)  +  P(0,l,t)B(0,l), 


(3) 


while  the  initial  conditions  for   (1)  and   (2)  are  taken  to  be 


P(m,n,t=0)  = 


ll   for  m=M,n=N, 
10   for  m^M,n^N. 


(4) 


Let  us  adopt  the  following  terminology  for  the  attrition-rate  co- 
efficients (and  hence  the  process  itself).   We  shall  say  that  we  have  a 
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(a)  linear-law  attrition  process  when 
A(m,n)  =  amn, 


B(m,n)  =  bmn,  (5) 


and 


(b)  square-law  attrition  process  when 
A(m,n)  =  |3m  +  an, 
B(m,n)  =  bm  +  an,  (6) 

where  a,  8  may  be  referred  to  as  operational  loss  rates. 

Although  it  is  well-known  that   (2)  through   (4)  yield  an 
exponential  solution  (the  well-known  Chapman-Kolmogorov  equation 
expresses  the  semi-group  property  of  the  state  probabilities  (see 
[29]))  when  A(m,n)   and  B(m,n)   are  specified  (for  example,  by 
(6)),  general  solutions  have  been  optained  to  this  system  only  in  a 
few  special  cases.   In  the  special  case  when  a  +  a  =  b  +  3,   Isbell 
and  Marlow  [27]  developed  a  general  solution  to   (2)  through   (4) 
for  a  square-law  stochastic  attrition  process.   Recently,  Clark  (see 
pp.  102-104  of  [9])  obtained  the  general  solution  to  the  linear-law 
stochastic  attrition  process  (i.e.   A(m,n)   and  B(m,n)   are  given 
by   (5)).   One  reason  why  we  have  reviewed  this  material  is  to  now 
point  out  to  the  reader  that  a  general  solution  to   (2)  through   (4) 
only  exists  for  a  linear-law  attrition  process  and  is  very  complex 
(see  pp.  102-104  of  [9]).   In  considering  the  optimal  control  ol  tl 
Lanchester  stochastic  (square-law)  process,  we  will  encounter  .1 
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similar  system  of  equations  for  the  optimal  expected-value  function. 
Keeping  in  mind  that  a  general  solution  to  the  corresponding  equations 
(2)  through   (4)  for  the  state  probabilities  of  the  square-law  sto- 
chastic attrition  process,  we  have  not  (seriously)  tried  to  develop 
a  general  solution  for  those  equations  satisfied  by  the  optimal  expected- 
value  function  in  view  of  such  difficulties. 

Additionally,  using  the  above  noted  solutions  for  the  Lanchester 
stochastic  process,  Clark  (following  results  in  [34]  and  qualitative 
results  in  [37])  made  a  comparison  of  the  average  force  levels  in  the 
stochastic  attrition  process  (denoted  as  m(t)   and  n(t))  and  the 
corresponding  force  levels   x(t)   and  y(t)   in  the  deterministic 
model  (such  as  (1)).   Unlike  the  corresponding  situation  for  the 
Yule-Ferry  linear  birth  process  (see  pp.  77-78  of  [2]  or  pp.  156-159 
of  [11]),  there  is  a  bias  (due  to  "boundary  effects")  in  dynamical 
behavior  of   x(t)   and  y(t)   as  compared  with  m(t)   and  n(t)   for 
the  same  a's   and  b's.   This  is  a  major  result  of  Clark's  careful 
investigation  in  which  several  numerical  examples  are  given  to  prove 
such  points.   Clark's  solution  to  the  stochastic  linear-law  process 
was  important  in  making  such  a  comparison.   This  fact  that  the  average 
of  the  Lanchester  stochastic  process  does  not  behave  identically  to 
the  corresponding  force  levels   x(t)   and.  y(t)   computed  according 
to  the  corresponding  deterministic  model  has  led  us  to  make  the  present 
investigation. 
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3.   The  Optimal  Control  Problem. 

In  Appendix  G  we  studied  the  following  optimal  control  problem 
for  the  deterministic  Lanchester  attrition  process: 

maximize  (ry(T)-px  (T)-qx0(T)}  with  T  =  T  specified, 
<Kt)  x      2  X 

,  .         dx, 
subject  to:  1  =  -  fay, 

dt        L 
dt  z 


dy_  =  -b  x  -b  x  ,  (7) 

dt  l 


x1,x2,y  ^  0        and        0  <.    4  <;  1, 
where 


x  (t),  x  (t)  ,  y(t)   are  force  levels, 


p,  q,  r  are  utilities  assigned  survivors, 


a  ,  a  ,  b  ,  b_   are  (constant)  attrition-rate  coefficients, 


and   4  is  the  fraction  of  Y  fire  directed  at   X..  , 


The  initial  conditions  for  the  above  problem  are 


x1(t=0)  =  x°, 
x2(t=0)  =  x2, 
y(t=0)  =yQ. 
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The  above  combat  scenario  is  for  a  prescribed  (fixed)  length  of  time 
T    (although  one  must,  of  course,  give  consideration  to  "premature" 
terminations  as  we  did  in  Appendix  G)  .   For  convenience,  we  have  chosen 
to  call  this  problem  the  "prescribed  duration  battle." 

Let  us  take  the  following  for  our  optimal  control  problem  in 
which  the  combat  is  described  as  a  stationary  Markov  process  (with 
a  discrete  state  space) : 


maximize  {m(T)-pm  (T)-qm?(T)}  with  T  =  T   specified. 

<Kt)         l  x 


subject  to:   casualties  occur  randomly  as  a  stationary 
Markov  process  corresponding  to  the 
deterministic  process  (7), 


(8) 


m  ,  m  ,  n  2  0 


and 


0  <;  <t  £  i, 


where 


m  (t),  m  (t),  n(t)   are  force  levels  (integers), 
p,  q,  r  are  again  utilities  assigned  survivors, 
a  ,  a„ ,  b  ,  b   are  (constant)  attrition-rate  coefficients, 
and  $      is  the  fraction  of  Y   fire  directed  at  X  . 


Furthermore,  <$>  (t)   is  restricted  to  take  on  values  from  the 

1   2 
set   {0,  n,  n,  .  .  . ,  l} . 


4.   Development  of  Fundamental  Functional  Equation. 

Let   S(x,m  ,m  ,n)   denote  the  optimal  expected-value  function 
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(see  [14]).   Then 


S(T,m1,m2,n)  =  S(x ,m  (t) ,m2 (x) ,n(x) ) , 


and 


S(T,mrm2,n)  =  ngx^mum  Bj 


rn (T=0)-pm1 (T=0)-qm2 (x=0) 


,  (9) 


where 


the  system  is  in  state   m  ,  m  ,  n   at  time   T   (i.e.  there  are 

m   of  the  X  -forces,  etc.), 
the  system  is  in  state  m  (x=0),   m  (t=0) ,   n(x=0)   at   T  =  0, 

$   is  the  class  of  admissible  controls  (i.e.   4>(t)   must  always 

1    2 
be  chosen  from  the  set  of  rational  numbers   {0, — -, — r  ,  \ 

'n(x),n(T), 

T  =  T  -  t   is  the  backwards  time  from  the  end  of  battle  (which 
begins  at   t=0) , 

E-*    denotes  mathematical  expectation  (with  respect  to  all 

m,T 

possible  states  at   T=0)  given  that   m(l)  =  (m  (T),m2(i), 
n(T)), 
casualties  occur  in  a  random  fashion  between   t  and  T. 


In  other  words,   S(l,m  ,m  ,n)   is  the  maximum  return  that  we  get  on 
the  average  when  we  start  with  force  levels   m  ,m  ,   and   n   at 
t  =  T-T,   follow  an  optimal  policy   4-*  (s)  (chosen  from  the  class  of 
admissible  policies   $)   for  t  <.    s  £  T,   and  casualties  occur  in  a 
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random  fashion. 

We  consider  that  casualties  occur  as  a  stationary  Markov  process 
with  discrete  state  space  (or  discontinuous  Markov  process).   Spe- 
cifically, we  assume  that 

(1)   the  attrition  process  is  a  stationary  Markov  process  corres- 
ponding to  a  deterministic  Lanchester  square-law  attrition 
process;  this  is  equivalent  to  assuming 

(a)  the  future  occurrences  of  casualties  depend  only  on  the 
state  of  the  system  at   t   and  not  on  past  history, 

(b)  the  transition  probabilities  depend  on  only  the  state 
of  the  system, 

(c) 


Prob 


Prob 


Prob 


one   X..   casualty 
in  interval   At 


one  X„   casualty 
in  interval  At 


one   Y   casualty 
in  interval  At 


=  <Ja,mAt, 


=  (l-<f)a2nAt, 


=  (b1m1+b2m2)At, 


where   (Ja  n   is   X   casualty  rate,  etc., 


(d)   Prob 


more  than  one  casualty 
in  interval  At 


=  0((At)2), 


where  following  Copson  (see  p.  35  of  [10])   0(x) 
denotes  dependence  on  x  such  that 
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T     0(X)  »        * 

ljyn  =  a  constant, 


(2)  the  Y-forces  have  perfect  information  as  to  the  state  of 
the  system  at   t  and  the  expected  casualty  rates, 

(3)  the  Y-forces  can  instantaneously  shift  fire  from  any  target 
at  any  time, 

(4)  the  length  of  the  battle  is  known. 
Further  defining  our  notation,  we  have 

m  (t),m  (t),n(t)   are  number  of  X  -forces,  X2~forces,  Y-forces, 
respectively,  as  a  function  of  time   t (restricted 
to  be  non-negative  integer) , 
and 
4(t)   is  fraction  of  Y-force  total  fire  directed  at  X^^    (restricted 
to  be  rational  between 
0  and  1  with  denominator  n(t)). 


Thus ,  we  have 


state  variables:   m  (t) ,m2 (t) ,n(t)  , 


decision  (or  control)  variable:   4(t), 


where 


4(t)  e  $  = 


°'n(t)'n(t)"-"  n(t)  *L 


To  be  more  precise   4  -  <  (n^.nyn.t)   is  a  closed-loop  (or  feedback) 
trol  (see  Appendix  F  for  a  further  discussion  and  contrast  with  open- 
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loop  control),  although  for  notational  convenience  we  don't  explicitly 
show  this  dependence  below.   Furthermore,  let   T   denote  the  "backwards 
time"  from  the  end  of  battle,  i.e.   T  =  T-t. 

To  develop  the  fundamental  functional  equation  for  the  optimal 
expected-value  function,  we  begin  by  considering  any  interval  of 
"backwards  time"  of  length  At  which  occurs  from  t-At   to   T.   There 
are  five  exhaustive  and  mutually  exclusive  possibilities  for  random 
events  to  occur  in  such  an  interval.   These  are 

(1)  one  X..   casualty  occurs, 

(2)  one   X„   casualty  occurs, 

(3)  one  Y  casualty  occurs, 

(4)  no  casualty  occurs, 

(5)  more  than  one  casualty  occurs. 

Let  us  now  examine  each  of  these  cases  and  develop  expected  returns. 
(1)   One  X..   casualty  occurs  in  At: 

By  our  assumptions  above,  we  have  for  the  probability 
of  occurrence  of  this  event 

Prob[one  X   casualty  occurs  in  At]  =  <ja  nAT .  (10) 

Given  that  one  X,   casualty  is  realized  in  the  interval  from  T   to 
T  -  At ,  the  optimal  target  selection  policy  for  Y  will  consider  the 
maximum  expected  value  for  the  return  functional  as  casualties  continue 
to  occur  randomly  from  T  -  At   to   T  =  0.   This  maximum  expected  value 
is   S(T-AT,m  (t-At) ,m2(T-AT) ,n(T-AT))   where  m  (t-At)  =  m  (t)-1, 
m-(T-AT)  =  m  (t),   and  u(t-At)  =  n(T). 
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(2)  One  X   casualty  occurs  In  At: 
Similarly,  we  have  that 

Prob[one  X2  casualty  occurs  in  At]  =  (l-(j:)a  n  At,        (11) 

with  the  optimal  expected-value  function   S(t-At  ,m,  (t)  ,m  (T)-l,n(T)  )  . 

(3)  One  Y  casualty  occurs  in  At: 
Similarly,  we  have  that 

Prob[one  Y   casualty  occurs  in  At]  =  (b  m  +b  m  )  At,      (12) 

with  the  optimal  expected-value  function   S(T-AT,m  (T),m  (T),n(T)-l). 

(4)  No  casualty  occurs  in  At: 

Prob[no  casualty  occurs  in  At]  = 

1  -  {(ja^+d-  cDa^+b^+b^}  At  +  0((At)2).  (13) 

The  optimal  expected-value  function  at   t-At   is  then  accordingly 
S(T-AT,m1(T),m2(T),n(T)). 

(5)  More  than  one  casualty  occurs  in  At: 

The  probability  that  this  event  occurs  is   0((At)  ). 

Now,  by  the  standard  dynamic  programming  argument  which  combines 
the  probabilities  of  events   (1)  through   (5)  above  with  the  maximum 
expected  return  to  be  achievable  given  these  events  occur,  we  obtain 
the  expression 
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S(t  ,m.  ,m_  ,n)  =  maximumi  [1-Al{(^a  n+(l- c{)a„n  + 
1  OS  fell 


blmi+b2m2^S^T~AT'ml'm2'n^  +  ^ain  AT  s(T_AT'm1~1'm2'n^ 


+  (l-<£)a_n  Ax  S(T-Ax,m  ,m»-l,n)  + 


(b  m  +b  m2)  At  S(t-At ,m  ,m2,n-l) } .     (14) 


Rearranging  terms  in  (14)  and  taking  the  limit  as  At-K),   we 
obtain  the  fundamental  functional  equation  for  the  optimal  expected- 
value  function   S(T,m  ,m  ,n) 

for  m  >  0,  m  >  0,  n  >  0: 
^|.(T,m1,m2,n)  =  (b^+b^)  {S (t  ,m  ,m2 ,n-l)  -  S  (t ,m  ,m2  ,n)  } 


+  n  maximum  [(fa  {  S  (t  ,m  -1  ,m_  ,n)  -  S(T,m1  ,m_  ,n)  } 


+  (l-<t)a2{S(x,m  ,m2-l,n)  -  S(T,m  ,m  ,n) }]  ,  (15) 

with  the  boundary  condition  at   t  =  T 

S(T=0,m  ,m2,n)  =  rn(x)-pm  (T)-qm2 (t) ,  (16) 

where 
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m  ,m  ,   and  n  are  integers, 
and 

$  =  {o  1  I     (n-1)   } 

The  special  forms  of  (15)  in  cases  when  m  =0,   etc.  will  be  given 
later. 

More  concisely,  we  could  have  said  that  (15)  results  from  com- 
bination of  the  well-known  formalism  of  dynamic  programming  with  the 
retrospective  (backward)  probabilistic  evolution  of  the  system  over 
time  (c.f.  [21],  [31]).   It  is  to  be  noted  that  (15)  is  a  special 
case  of  an  equation  given  by  Kushner  in  1962  [30]. 

If  we  take  (15)  to  be  the  fundamental  (or  basic)  equation  for 
S(i,m  ,m  ,n),   then  (14)  may  be  considered  to  be  the  simplest  finite 
difference  approximation  to  it,  i.e.  the  result  of  applying  the  well- 
known  Euler's  method  to  (15)  (see  pp.  130-131  of  [53]).   (Of  course, 
a  method  employing  a  higher  order  approximation  scheme  (see  pp.  132- 
140  of  [24])  may  be  necessary  under  many  circumstances.)   We  will 
find  this  point  of  view  convenient  when  we  consider  developing  a 
solution  to  (15). 

Finally,  it  seems  appropriate  for  us  to  note  that  equations 
(15)  have  precisely  the  same  structure  as  the  equations  for  the 
Lanchester  (square-law  attrition,  since  the  attrition  rates  have  the 
same  form  as  (6))  stochastic  process  for  which  (to  the  best  of  our 
knowledge)  no  general  solution  has  been  developed.   This  last  fact 
shows  that  although  it  is  a  well-known  and  trivial  theoretical  result 
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that  the  solution  to  such  a  first  order  (finite)  system  of  linear, 
constant-coefficient  ordinary  differential  equations  is  given  by  the 
matrix  exponential,  there  may  be  practical  difficulties  in  obtaining 
a  more  explicit  representation  for  computing  numerical  values  of  the 
solution,  especially  when  the  number  of  equations  in  the  system  is 
variable  (here  depending  on  three  parameters). 

5 .   On  the  Analytic  Solution  to  the  Fundamental  Functional  Equation. 

The  first  task  in  determining  the  optimal  target  selection 
policy  (which  requires  obtaining  the  solution  to  (15)  and  (16))  is 
to  develop  the  entire  system  of  equations  (c.f.  equations  (2)  through 
(4)).   We  must,  therefore,  consider  the  form  that  (15)  takes  at  the 
boundary  of  the  system,  i.e.   m  =0  or  m  =0  or  n=0,   where  the 
target  selection  problem  no  longer  exists.   When  n=0,   arguments 
similar  to  the  above  lead  to 


for  n  =  0,  m  ^  0,  nu  2:  0, 


— (x,m1,m2,0)  =  0  with   S (T=0,m  ,m2 ,0)  =  -m  p-m  q. 


Hence, 


for  n  =  0,  m  2:  0,  nu;>0, 


S(x,m  ,m  ,0)  =  -  m  p  -  m  q  (17) 


Similarly, 
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for  m  =  0,  m2  =  0,  n  ;>  0, 

S(T,0,0,n)  =  nr,  (18) 

for  m  =  0,  m  >  0,  n>0, 

J  C 

^r(T,0,m2,n)  =  b^  {S  (t  ,0,m2  ,n-l)  -  S(x,0,m2,n)} 

+  a2n{S(T,0,m1-l,n)  -  S (t ,0,m  ,n)  },  (19) 

for  m>0,m=0,n>0, 

— (T,m1,0,n)  =  b1m1{S(T,m1,0,n-l)  -  S(i,m  ,0,n)} 

+  a1n{S(T,m1-l,0,n)  -  S (t ,m  ,0,n) } .  (20) 

Equations  (15)  through  (20)  are  the  complete  system  of  equations  for 
the  optimal  expected-value  function  in  the  optimal  control  of  the 
Lanchester  stochastic  process. 

For  m  >  0,  m   >  0,  n  >  0   the  optimal  target  selection  policy 
is  determined  by  the  maximization  operation  in  (15),  and  hence 


C?*(T)  = 


1   for  W(l,m  ,m  ,n)  >  0, 

0   for   W(x,m  ,m  ,n)  <  0,  (21) 


where  we  shall  refer  to   W(x,m  ,m  ,n)   as  the  "switching  function." 
It  is  defined  by 


for  m  >  0,  m  >  0,  n>0, 
W(x,m  ,m  ,n)  =  a  { S (x ,m  -1 ,m2 ,n)  -  S(x .m^ ,m2 ,n) } 

-  a2(S(x,m  ,m2-l,n)  -  S (x ,m1 ,m2 ,n) } .  (22) 
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Let  us  observe  that  at  the  end  of  the  battle  at   t  =  T,   we  may 
combine  (16),  (21),  and  (22)  to  obtain 


cj*(T=0)  = 


1   for   a  p  >  a2q, 

0   for  a±p   <  a2q,  (23) 


which  the  reader  will  recall  as  the  same  result  previously  obtained 
for  the  optimal  control  of  the  deterministic  process  (see  Appendices 
A,  F,  and  G) . 

As  pointed  out  in  Section  4,  equations  (15)  through  (20)  have 
the  same  form  as  the  equations  for  the  Lanchester  square-law  attrition 
stochastic  process  (i.e.  equations  (2)  through  (4)  when  the  attrition 
rates  are  given  by  (6)).   (We  note  that  (15)  reduces  to  a  first  order 
system  of  ordinary  differential  equations  for   S(x,m  ,m  ,n)   when 
(j)*  as  determined  by  (21)  is  used.   Solving  for   S(x,m  ,m_n)   for 
m  =  0,1,2 ,...  ,etc.  ,   we  can  then  determine   ({*  by  (21).   The 
synthesis  of  optimal  control  by  combination  of  the  control  law  (21) 
with  integration  of  a  system  of  differential  equations  is  similar  to 
that  for  deterministic  optimal  control  problems.)   We  have  pointed 
out  that  (to  the  best  of  our  knowledge)  a  general  solution  has  not 
been  obtained  for  the  Lanchester  stochastic  process  equations.   Hence, 
it  seems  unlikely  that  an  analytic  solution  is  readily  obtainable 
for  (15)  through  (20)  (although  such  a  solution  may  be  considered 
to  be  a  matrix  exponential  (see  above)). 

Nevertheless,  it  is  of  value  to  develop  a  partial  solution.   For 
example,  since  we  use  finite  difference  methods  to  generate  an  approxi- 
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mate  solution,  it  is  desirable  to  check  the  adequacy  of  the  approxi- 
mation (in  particular,  the  "time  step  size"  used  in  the  numerical 
propagation  of  the  approximate  solution  by  "marching  ahead  in  time") 
This  is  easily  done  by  comparing  the  approximate  solution,  denoted 
by  S,   to  the  exact  analytic  solution,  denoted  by   S.   Hence,  a 
partial  analytic  solution  is  useful. 

Careful  consideration  of  (15)  through  (20)  reveals  that  there 
are  restrictions  on  the  order  in  which  the  optimal  expected-value 
functions   S(i,m  ,m„,n)   for  m  =0,1,2, ... ,etc.   can  be  computed. 
In  particular,  one  admissible  sequence  for  building  up  the  solution 
through   S (1,1,1,1)   is  shown  below  in  Table  1. 


ml 

m2 

n 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

1 

1 

1 

0 

0 

1 

1 

1 

0 

1 

1 

1 

1 

Table  I.   Admissible  Order  for  Computing  Optimal  Expected-Value 
Functions  (admissible  order  is  from  top  to  bottom) . 


Thus,  we  readily  successively  compute  using  (17)  through  (20) 

S(i, 0,0,0)  =  0, 
S(i, 1,0,0)  =  -p, 


S(t, 0,1,0)  =  -q 
S(t,  0,0,1)  =  r, 
S(t,  1,1,0)    =    -p-q, 


/b   r-a   q\      -(a  +b    )t        /a   r-b   q\ 

s(TMi,=Me      +(w 

/b   r-a   F\     "U1+b  1)T        /alr~blP\ 

s(Tii.o1i)-U-55y.  +(-^bH-         (24) 


Furthermore,  for  m..  =  1,  m  =  1 ,  n  =  1,   by  (24)  equations  (15)  and 
(16)  become 

||(t, 1,1,1)  =  -(b1+b2){S(T, 1,1,1)  +  (p+q)} 

+  maximum  [<j>a..  {S(t  ,0  ,1,1)  -  S(t, 1,1,1)}  + 

0  £  $  £  1 

<j>  =  0  or   1 

(l-<)>)  a2(S(T, 1,0,1)  -  S(x, 1,1,1)}], 

with  S(t=0,  1,1,1)  =  r  -  p  -  q,  (25) 

where  S(x ,0,1,1)   and  S(x, 1,0,1)   are  given  by  (24).   It  is  con- 
venient to  denote  that  "switching  function"  as  w(x) 

w(t)  =  W(t, 1,1,1)  = 

a^Sd, 0,1,1)  -  S(x, 1,1,1)}  -  a2{S(T, 1,0,1)  -  SCt, 1,1,1)},  126) 
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so  that  (21)  becomes 


4>*(t) 


'   1   for  w(t)  >  0, 

0  for  wCt)  <  0.  (27) 


We  observe  that 


w(t=0)  =  a^p  -   a2q,  (28) 

so  that  (23)  continues  to  hold. 

Using  (23),  (24),  (26),  and  (27),  we  may  readily  solve  (25). 
As  in  the  deterministic  case  (see  Appendix  G) ,  there  are  two  cases 
that  must  be  distinguished 


Case  (1)   a  p  ^  a  q, 
Case  (2)   a  p  <  a2q. 

For  Case  (1):   a  p  ^  a„q ,   we  have  that   $*(t)  =  1  for 
0  ^  t  £  x  ,   where   x   denotes  the  "backwards  time"  of  the  first 
switch  in  the  optimal  target  selection  policy,  t^   is  the  smallest 
t  which  satisfies  w(t=t  )  =  0  with  w(x)   given  by  (26). 

for  0  £  x  ^  t,   when   a  p  ^  a~q  (<t>*(x)=l) 

/,       x     -(a~+b0)i 
S(t,1,1,1)  -    a^b^-a^)     £   2   2   + 

(a  +b  -a2) (a2+b2) 
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[(b1-a2)(b1+b2)+a1b1]r 


alP 


(a1+b1-a2) (a1+b1+b2) 


+ 


axa2q 


(a1+b1+b2)         (a1+b1-a2)(a1+b1+b2) 


e 


-(a^b^b^i 


axa2r 


(a2+b2) (a+b1+b2) 


(b1+b2)p 
(a1+b1+b2) 


[Cb1+b2)(a2+b2)+a1b2]q 
"Ta^+b^Ka^+b^+b^) 


(29) 


We  note  that   t1   might  be  equal  to  +  °°,   i.e.  we  never  switch. 
Assuming  that  a  switch  in  targets  does  occur,  however,  let  us  denote 
S(t=t1 ,1,1,1)   by   S   where,  as  we  recall,   x1   is  the  smallest 
x  which  satisfies  w(t=t  )  =  0.   Then,  we  have  that   <J>*(x)  =  0 
for  x,  <  x  i»  t  ,  where   x   denotes  the  "backwards  time"  of  the 
second  switch  in  the  optimal  target  selection  policy.   Then,  we 
have 

for      x      <   x  ^    x       when     a  p  ^    a„q    (cj>*(x)    =   0) 


a2(blr~alp) 


S(t, 1,1,1)         (a2+b2_ai)(ai+bi) 


-(a1+b1)x      (a2+b2)  (x1-x)-a1xn-bnx] 
e  -e 


rT)-aiTrV] 


ala2r 
S0        (a1+b1)(a2+b2+b1) 


+ 


Kb1+b2)(a1+b1)+a2b1]p 
(a1+b1) (a2+b2+b1) 


(b1+b2)q  |    ( 
(a2+b2+b1)Je 


a2+b2+bi)  (xrx) 


+ 


ala2r 


(a1+b1) (a2+b2+b1) 


[(b1+b2)(a1+b1)+a2b1]p    (b1+b2)q 


(a1+b1)(a2+b2+b1) 


(a2+b2+b1) 


(30) 
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Again,  we  note  that  x^     might  be  equal  to  +  °°,   i.e.  we  might 
never  switch  targets  a  second  time.   Assuming  that  a  second  switch 
in  targets  does  occur,  we  have  4>*(x)  =  1  for  t  <  t  si  t  . 
We  have  not  carried  the  computation  of   S(t, 1,1,1)   past   x^. 

For  Case  (2):   a^p  <  a2q ,  we  have  that  <£*(t)  =  0  for 
0  £  t  £  t^,  where  t.   denotes  the  "backwards  time"  of  the  first 
switch  in  the  optimal  target  selection  policy,   x   is  the  smallest 
t  which  satisfies  w(t=t1)  =  0  with  w(t)   given  by  (26). 

for  0  £  t  <:  t   when  a  p  <  a  q   (<j>*(x)  =  0) 


a2(b1r-a1p)     -(a^b^x 

s(t, 1,1,1)  =  ,      —   rmn.  ^  e      + 

^2   2~  1'  ^  1   r 


[(b2-a1)(b1+b2)+a2b2]r 
(a2+b2-ai) (a2+b2+b1) 


+ 


ala2P 


a2q 


(a2+b2~a1)  (a^b^b.^ 


(a2+b2+b1) 


-(a2+b2+b1)x 


a  a  r 


(a1+b1) (a2+b2+b1) 


[(b1+b2)(a1+b1)+a2b1]p 
(a1+b1) (a2+b2+b1) 


(b1+b2)q 


(a2+b2+b1) 


(31) 


We  note  that   x   might  be  equal  to  +  °°,   i.e.  we  never  switch. 
Assuming  that  a  switch  in  targets  does  occur,  however,  let  us  denote 
S(x=x  ,1,1,1)   by   S„  where,  as  we  recall,   x.   is  the  smallest 
t  which  satisfies  w(x=x  )  =  0.   Then,  we  have  that   4>*(x)  =  1 
for  x   <  x  ^  x„,  where   x_   denotes  the  "backwards  time"  of  the 
second  switch  in  the  optimal  target  selection  policy.   Then,  we 
have 
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for   x   <  x  £  x2  when  a  p  : 


S(t, 1,1,1)  = 


ai(b2r-a2g)    ' 
(a1+bra2)  (a2+b2) 


r  -(a2+b2)x   Ca1+b1)(r  -x)-a2T  -b2t 
e         -e 


+ 


S„  - 


aiV 


O^+b^p 


0   (a1+b1+b2)(a2+b2)  +  (a1+b1+b2)  ' 


[(b1+b2)(a2+b2)+a1b2]q 
(a1+b1+b2)(a2+b2) 


(a1+b1+b2)(t1-T) 


+ 


aia2r 


(bj+b2)p 

(a1+b1+b2) (a2+b2)  "  (a1+b1+b2) 


[(b1+b2)(a2+b2)+a1b2]q] 


(a1+b1+b2) (a2+b2) 


•  (32) 


Again,  we  note  that   x_  might  be  equal  to  +  °°,   i.e.  we  might 
never  switch  targets  a  second  time.   Assuming  that  a  second  switch 
in  targets  does  occur,  we  have   4>*(t)  =  0  for   t„  <     x~. 
have  not  carried  the  computation  of   S(x, 1,1,1)   past   x«. 

Although  the  above  constitutes  a  complete  solution  for 
and  hence  <}>*(x)   via  w(x)  ,   these  results  are  complex  < 
it  is  not  immediately  clear  how  <j>*(x)   changes  over  time  and/ 
depends  on  the  variable  model  parameters.   (We  recall  that  in  the 
deterministic  case  when  x..  (T)  >  0   and  x?(T)  >  0,   the  conditions 

alP  ^  a2q   and   albl  >  a2b2   imPlied  tiiat  (I)*(T)  =  1   fo 

entire  battle.)   We  plan  to  analyze  such  aspects  more  thoroughly 
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both  analytically  and  computationally  in  the  future. 

6.   Development  of  Numerical  Solution. 

As  we  have  already  discussed  several  times  above,  a  general 
solution  for  S(x,m, ,m  ,n)   which  satisfies  (15)  through  (20)  for 
m  ^  0,  m?^  0,  n  ^  0  has  not  been  obtained  and  prospects  do  not 
seem  bright  for  this.   With  the  advent  of  modern  high-speed 
digital  computers,  finite  difference  methods  of  obtaining  an 
approximate  solution  are  commonly  used  under  such  circumstances. 

Euler's  method  (see  pp.  130-131  of  [24])  yields  the  simplest 

finite  difference  approximation  for  (15) .   Let  us  denote  the 

approximation  to  the  optimal  expected-value  function  by  S. 
We  shall  compute  values  for  the  approximation  to  the  optimal 

expected-value  function  at  discrete  points  in  time  separated  by  a 

constant  amount   At.   We  let   t  =  £At   so  that  T  =  LAt .   Then 

(15)  may  be  approximated  by 

for  el  >  0,  m0  >  0,  n  ;>  0: 

S((Jl+l)AT,m1,m2,n)  =  (l-(Ax)  (b-^+b^)  }  S(£At  ,m1  ,m2  ,n) 

+  (Ax)(b1m  +b2m  )  S(£At  .m^m^n-l) 

+  y(Ax)   maximum   [<}>a,  {S(£Ax  ,m.-l  ,m_  ,n) 
0  £  4>  £  1 

4>  €  $ 

-  S(^AT,m1,m2,n)}  +  (I-4.)  a2{S(£Ax  ,11^,111-1  ,n)  -  S(£Ax  ,m  >m2  ,n)  }  ]  ,  (33) 
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for  I   =  0,1,..., L  -  1  with  boundary  condition 


S(0,m1,m2,n)  =  rn  -  pm  -  qnu , 


where 


m1  ,  m  ,   and  n  are  integers 


and 


n  n' 


(n-1) 


We  earlier  had  seen  how  (33)  arose  in  the  derivation  of  (15) . 
Similar  approximations  may  be  developed  for  (19)  and  (20) . 

As  we  noted  above,  consideration  of  (15)  through  (20)  yields 
that  there  are  restrictions  on  the  order  in  which  the  optimal 
expected-value  function  S   (or  its  approximation   S) .   We  observe 
that  the  computation  S( (£+1) At ,m1 ,m  ,n)   depends  upon 


S(AAT,m.,iii  n)  , 
S(£AT,m  -l,m  ,n) , 
S(ZAT,m  ,m  -l,n) , 
S(£At ,m1 ,m  ,n-l) . 


The  dependence  upon  the  state  variables  is  shown  in  Figure  1 
below. 
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m  -l,m~ ,n 


\ 


\ 


m  ,m. ,n-l 


/ 


y 


m  ,m  ,n 


Figure  1.   Dependence  of  Optimal  Expected-Value  Function 
on  Discrete  State  Variables. 


Based  on  the  dependence  shown  in  Figure  1,  the  solution  can 
be  !,built-up"  as  shown  in  Table  II.   the  general  case  is  shown 
in  Table  III. 

It  remains  to  discuss  the  adequacy  of  the  finite  difference 
approximation  (33).   It  is  well-known  (see  pp.  130-145  in  [24]) 
that  Euler's  method  yields  a  finite  difference  approximation  for 
such  a  system  of  differential  equations  that  is  both  consistent 
and  stable  so  that  the  approximate  solution  S   can  be  guaranteed 
to  converge  to  the  exact  analytic  solution   S  as   Ax  ■*  0   (and 
L  ->  °°)   [36].   However,   At  must  not  be  too  large  in  order  to 
keep  the  truncation  error  satisfactorily  small.   Moreover,  the 
time  step  size   At   is  also  limited  by  the  fact  that  a  quantity 
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Table  II.   Admissable  Order  for  Computing 

Optimal  Expected-Value  Functions 


=1 

nL2 

n 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

1 

1 

1 

0 

o 

1 

1 

1 

0 

1 

1 

1 

1 

2 

0 

0 

0 

2 

0 

0 

0 

2 

2 

1 

0 

1 

2 

0 

2 

2 

0 

0 

2 

1 

0 

1 

2 

0 

2 

2 

2 

0 

1 

1 

0 

2 

2 

0 

2 

2 

1 

1 

1 

2 

1 

1 

1 

2 

2 

2 

1 

1 

2 

2 

2 

1 

2 

2 

2 

2 

3 

0 

0 

0 

3 

0 

0 

0 

3 

Si 

m2 

n 

3 

1 

0 

3 

2 

0 

1 

3 

0 

2 

3 

0 

3 

3 

0 

0 

3 

1 

0 

3 

2 

0 

1 

3 

0 

2 

3 

0 

3 

3 

3 

0 

1 

3 

0 

2 

1 

0 

3 

2 

0 

3 

3 

0 

3 

3 

1 

1 

3 

2 

1 

3 

1 

2 

3 

2 

2 

1 

3 

1 

2 

3 

1 

1 

3 

2 

2 

3 

2 

1 

1 

3 

2 

1 

3 

1 

2 

3 

2 

2 

3 

3 

3 

2 

2 

3 

3 

3 

2 

3 

3 

3 

3 

A 

0 

0 

0 

4 
etc. 

0 

Note:   Admissible  order  is  top  to  bottom,  starting  with  column 
(composed  of  m..  ,  m  ,  n)   on  left. 
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Table  III.   General  Case  for  Admissable  Order 

of  Computing  Optimal  Expected-Value  Functions. 


% 

^2 

n 

i 

0 

0 

0 

i 

0 

0 

0 

i 

i 

1 

0 

i 

• 

2 

• 

0 

• 

• 

i 

• 

(i-D 

• 

0 

1 

i 

0 

2 

i 

• 

0 

• 

• 

(i-D 

• 

i 

• 
• 

0 

i 

i 

0 

0 

i 

1 

0 

• 

i 

• 

2 

• 

• 

0 

• 
• 

i 

• 

(i-D 

0 

1 

i 

0 

2 

i 

• 

• 
• 

• 
• 

0 

(i-D 

• 

i 

0 

i 

i 

i 

0 

1 

i 

0 

2 

(i-D 


Si 

2*2 

n 

1 

0 

i 

2 

• 

0 

• 

i 

• 

(i-D 

• 
• 

0 

• 
• 

i 

i 

0 

i 

i 

1 

1 

i 

• 

2 

1 

• 

• 
• 

i 

(i-D 

1 

i 

1 

2 

i 

• 

2 

• 

2 

• 

• 

i 

• 

(i-D 

• 

2 

i 

• 

1 

• 

3 

• 

• 
• 

i 

(i-D 

(i-D 

1 

i 

1 

2 

• 

i 

• 

1 

• 

• 

(i-D 

• 
• 

i 

• 

(i-D 

1 

1 

i 

2 

1 

i 

• 

• 
• 

• 
• 

(i-D 

(i-D 

• 
i 

i 

i 

(i-D 
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Note:   Admissible  order  is  top  to  bottom,  starting  with  column 
(composed  of   m  ,  nu ,  n)   on  left. 
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like   (At) (b  m  +b?m„)   or   a  nAx   or  a  nAx   in  (33)  represents 
a  probability  and  hence  must  be  less  than  one.   Thus,  in  compu- 
tational work  we  must  place  the  following  restrictions  on  the 
time  step  size 


^blmi+b2in2^AT   <   1> 


a  nAx   <    1, 


a„nAx    <   1.  (34) 


Moreover,  in  our  computational  work  (performed  by  M.S.  thesis 
student,  R.  Powers,  Lt.  USN)  we  have  used  a  time  step  size  which 
yields  agreement  in  the  fourth  decimal  place  to  the  right  of  the 

A 

decimal  point  when   S   is  compared  to  the  exact  analytic  solution 
S   in  the  special  cases  (such  as  (24)  through  (32)  when  the  latter 
has  been  obtained. 

7 .   Solution  Behavior  for  Large  Number  of  Combatants. 

Analytically  we  have  developed  a  solution  to  equations  (15) 
through  (20)  up  through  S(x, 1,1,1)   (see  (24)  and  (29)  through 
(32) ;  see  also  Table  II  for  admissible  order  of  computation) . 
Computationally,  we  can  generate  a  numerical  solution  up  through 
S(x ,10 ,10,10)   (computed  from  (33)  and  requiring  computation  in 
the  order,  for  example,  shown  in  Table  III).   However,  it  appears 
infeasible  for  computational  reasons  (depending  upon  both  the 
amount  of  core  storage  and  also  CPU  processing  time)  to  generate 
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a  solution  much  past  force  levels  of  the  approximate  magnitude 
m  =  10,  m„  =  10,  n  =  10.   Thus,  it  is  of  import  to  consider 
approximate  solution  behavior  of  (15)  through  (20)  for  large 
numbers  of  combatants. 

We  will  first  consider  limiting  behavior  of  (15)  through  (20) 
by  (naively)  replacing  differences  in  force  levels  by  their 
approximations  as  partial  derivatives  times  the  length  of  the 
interval  (unity) — deliberately  ignoring  the  fact  that  S(t,x  ,x„,y) 
had  originally  been  defined  only  for  integer  values  of  x  ,  x_ 
and  y.   We  seek  to  approximate  our  problem  with  a  discrete  state 
space  by  one  in  which  the  state  of  our  system  (i.e.  numbers  of 
X  ,  X? ,   and  Y   combatants)  varies  continuously.   Markov  processes 
in  which  only  continuous  changes  in  state  occur  are  called  dif- 
fusion processes  (see  p.  203  of  [11]).  We  will  show  that  the 
naive  passage  in  the  limit  leads  to  an  inappropriate  equation 
(one  which  ignores  the  probabilistic  nature  of  the  process) .   Later 
we  shall  give  the  parabolic  partial  differential  equation  (see 
pp.  57-77  of  [22])  which  we  feel  yields  the  appropriate  approxi- 
mation to  the  fundamental  equation  (15) . 

Diffusion  approximations  to  Markov  processes  with  discrete 
states  have  been  extensively  studied  the  past  30  years  (see  survey 
article  by  Iglehart  [26],  sections  in  [2],  [11],  and  [18],  and 
the  papers  [19],  [23],  and  [28]).   We  will  give  some  of  our  pre- 
liminary results  below.   Our  arguments  will  be  formal,  with  a 
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rigorous  justification  being  beyond  the  scope  of  our  present 
investigation.   (In  such  matters  a  rigorous  justification  requires 
delicate  and  subtle  arguments  and  is  itself  a  topic  of  contemporary 
mathematical  research.)   The  idea  of  considering  diffusion  approx- 
imations to  Lanchester-type  stochastic  processes  is  due  to  D.  Gaver. 
(Of  course,  all  inadequacies  of  the  present  presentation  are  my 
own  responsibility.)   Diffusion  approximations  have  proven  to  be 
very  useful  in  genetics  [17] ,  [28]  ,  queueing  [23]  ,  and  other 
areas  of  applications  of  stochastic  processes. 

Let  us  first  exhibit  the  unsatisfactory  approximation  to  (15). 
The  first  thing  that  we  will  do  is  to  replace   t  by  T  -  t. 
Next,  we  replace  differences  in  force  levels,  for  example,  as 
follows 

S(t,mi,m2,n-1)  -  S(t,m1,m2,n)  =  {(y-l)-y)  ||(t ^ ,x2  ,y)  +  0((Ay)2),  (35) 

where  Ay  =  1.   Now,  for  large  numbers  of  combatants  we  may  ignore 
the  integer  restrictions  on  the  X,-,  X~-,   and  Y-  force  levels 
to  obtain  the  following  approximation  to  (15) 


9S    ru        u.u    ^  3S 

—  -  (b  x..+b0x0)  —  +  maximum 

3t     i  1   2  2   9y   Q  ^   ^ 


/♦yl-I^W^vi-^-0-  (36) 


where  S  =  S(t,x, ,x  ,y).   The  above  is  precisely  the  Hamilton-Jacobi- 
Isaacs  equation  for  the  deterministic  optimal  control  problem  (see 
Appendix  G) .   Similar  approximations  for  the  Lanchester  stochastic 
process  have  been  given  by  Willard  [45],  Koopman  [29],  and  Etter  [15], 
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We  will  show  below,  however,  that  the  above  approximation  is 
inappropriate,  since  the  corresponding  attrition  process  is  deter- 
ministic and  we  have  thusly  destroyed  the  probabilistic  nature  of 
the  attrition  process. 

We  now  show  the  equivalence  of  (36)  with  some  well-known 
results  from  the  optimal  control  of  the  deterministic  Lanchester 
attrition  process.   Clearly,  (36)  yields  the  maximum  principle 


maximum  H(t  ,x.*  ,p  ,*,<t>)  ,  (37) 

0  £  <}>  £  1     x       x 


where 


H(t,x.,p.  ,<j>)  =  -  (fia^pj^  "  (I"*)  a2yp2  -  (b^+b^)  p3> 


and 


Pi*(t)  =  H-(t,X;L,x2,y)   for   i  =  1,2, 
i 


P3*(t)  =  37(t'Xl'X2'y) 


3y 

Denoting  the  value  of   <f>  which  yields  the  maximum  in  (37)  by  4>* , 

we  obtain  the  following  partial  differential  equation  for   S(t .x^x^y) 

|f  -  (Vl*+b2x2*>  If  -  ♦  ♦.^  -  d-**)a2y*  g.  =  0>     (38) 

which  is  readily  solved  by  the  method  based  on  characteristics  (see 
[22],  especially  pp.  44-49)  to  yield 
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dx±* 


*any* 


dt      r   1 


dx  * 

_i_  =  _  (l-<j)*)a2y*, 


ff  =-(blXl*+b2x2*), 


dpi*      3H 
and  -^-  =   ~  J^   (t,x.*,p.*,4>*)   for   i  =  1,2,3,  (39) 


where,  for  convenience,  we  have  let   x„  =  y.   By  the  above,  we  see 
that  (36)  apparently  corresponds  to  a  deterministic  attrition 
process. 

To  more  fully  justify  our  claim  that  (36)  is  an  inappropriate 
diffusion  approximation,  it  is  necessary  to  review  some  material 
from  the  theory  of  stochastic  processes.   We  do  this  in  the  sub- 
sections below.   First,  we  discuss  a  diffusion  approximation  to  a 
simple  random  walk.   Next,  we  apply  such  considerations  to  the 
Lanchester  stochastic  process.   Finally,  we  present  our  diffusion 
approximation  to  the  fundamental  functional  equation  (15)  for  the 
optimal  expected  value  function.   The  diffusion  approximation  to 
the  Lanchester  stochastic  process  is  a  new  result,  previously  not 
appearing  in  the  literature. 

7.1.   Diffusion  Approximation  to  a  Simple  Random  Walk. 

It  seems  appropriate  to  begin  by  giving  our  own  treatment  of 
the  diffusion  approximation  to  the  simple  random  walk  given  by 
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Feller  (see  pp.  323-327  in  [18]).   Our  brief  treatment  is  based  on 

the  theory  of  first  order  partial  differential  equations  (see  pp. 

18-23  of  [22])  (we  refer  the  reader  to  Feller's  lucid  treatment  [18] 

for  a  development  based  on  considerations  of  random  variables)  and 

explicitly  shows  that  a  naive  passage  in  the  limit  can  destroy  the 

probabilistic  nature  of  the  model. 

Let  us  therefore  consider  an  unrestricted  random  walk  starting 

at  the  origin.   We  let  p     denote  the  probability  that  the 

x,n 

particle  is  at   x  at  time  n.   During  each  time  period,  the  par- 
ticle has  a  probability  p   of  moving  one  unit  to  the  left  and  q 
of  moving  one  unit  to  the  right  (where  p  +  q  =  1) .   The  following 
partial  difference  equation  describes  this  random  walk 


Px,n+1  =  P  Px-l,n+q  Px+l,n'  (40) 

with  initial  conditions 

P0;0  -  1    •   Px.O  "  °   for   X  *   °- 
Let  us  write  (40)  as 

p(x,n+l)  =  p  p(x-l,n)  +  q  p(x+l,n),  (41) 

with  initial  conditions 

p(0,0)  =  1,  p(x,0)  =0   for  x  4   0. 
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We  observe  that  the  time  parameter  is  discrete  (i.e.   n  =  0,1,2,...) 
and  the  state  space  is  discrete  (i.e.   x  =  -  °°, .  . .  ,-2  ,-1,0,1 ,2  , .  .  . )  . 

In  developing  a  diffusion  (Markov  process  with  continuous 
state  space  [11]  approximation  to  (41)  we  let  the  time  parameter 
have  increments  of  At   and  the  state  space  have  increments  of 
Ax  and  pass  in  the  limit  as   At  ->  0  and  Ax  ■*■   0.   Then,  (41)  may 
be  re-written  as 

p(x,t+At)  =  p  p(x-Ax,t)  +  q  p(x+Ax,t).  (42) 

If  we  approximate  (42)  by  retaining  only  first  order  terms,  we 
obtain 

|£  At  =  (Ax)(q-p)  |£.  (43) 

Letting  lim   (q-p)(-~)  =  -  2c, 

Ax  ^  0      V   ' 
At  -*  0 


we  may  obtain  from  (43) 


^£+  2c  -^  =  0  (44) 

3t    C  3x   U' 


with  initial  condition 


p(x,t=0)  =  fi  00  , 


or,  more  generally. 


p(x,t=0)  =  6 (x-x0) , 
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where   6(x)   denotes  the  delta  function  of  P.  Dirac.   Equation  (44) 
is  our  naive  approximation  to  the  random  walk  described  by  (40). 
The  quantity  p(x,t)   is  the  probability  density  of  the  particle, 


i.e. 


p(x,t)dx  =  Prob[particle  between  x  and   x+dx  at   t]. 

The  solution  to  (44)  is  readily  obtained  by  characteristics 
[22]  to  be 


dt   dx   dp 
1    2c  ==  0  ' 


or 


dx 
p(x,t)  =   constant   for  -r-   =  2c, 

dt 


which  we  may  write  as 

p(x,t)  =  6(x-xQ-2ct).  (45) 

Thus,  our  random  walk  has  become  a  completely  deterministic  process. 
We  may  think  of  this  as  that  all  probability  remains  concentrated 
at  a  point  which  moves  in  a  strictly  deterministic  fashion. 

Feller  (pp.  323-327  in  [18])  shows  that  an  approximation  to 
(41)  which  preserves  the  probabilistic  nature  of  the  random  walk 
process  is  given  by 

|E«-2c|E-+d%  (46) 

3t        9x     „  2' 
9x 
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with  initial  condition 

p(x,t=0)  =  6(x-x  ) . 

Equation  (46)  is  a  parabolic  partial  differential  equation  with 

solution 

2     1 


P(x,t)     =    rr-pr       exp 

(4^Dt)i/Z 


0c-xQ-2ct) 

4DF 


(47) 


In  (46)  ,   2c   is  termed  the  infinitesimal  mean  (or  drift)  and   2D 
is  the  infinitesimal  variance.   We  observe  that  (44)  is  a  special 
case  of  (46)  with  an  infinitesimal  variance  of  zero,  i.e.   D  =  0. 
Hence,  our  statement  that  (44)  represents  a  purely  deterministic 
process . 

Additionally,  it  seems  appropriate  to  point  out  that  there  is 
a  gap  in  Feller's  argument  on  p.  325  of  [18].   We  feel  that  he 
should  have  said  that  to  obtain  reasonable  results  one  must  let 
Ax  and   At   approach  zero  in  such  a  way  that  the  limit  of  the 
variance  of  the  total  displacement  in  time   t   is  a  finite, 
non-zero  quantity.   Feller  neglects  to  state  the  key  aspect  that 
the  infinitesimal  variance  be  non-zero.   Otherwise,  as  we  saw 
above,  one  can  easily  erroneously  obtain  an  approximating  equation 
which  may  be  thought  of  as  representing  a  purely  deterministic 
process  with  no  random  fluctuations  in  the  sample  path. 

Equation  (46)  is  a  special  case  of  the  forward  Kolmogorov 
equation  (also  called  the  Fokker-Planck  equation) 
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2 

Tt  =  T  — 2  {a(x)P}  ~  f^  <B(x)p},  (47) 

3x 


where 


P  -  p(t,x;tQ,x  )   is  the  transition  probability  density 


i.e.   p(t,x;tQ,x  )  dx  =  Prob 


particle  between  x 
and  x  +  dx  at  t 


particle  between  x. 
and  x.  +  dx   at  t 


We  recall  that  the  position  of  the  particle  is  a  random  variable, 
denoted  by  X(t) .   Also,   g(x)   is  the  infinitesimal  mean  and  is 
given  by 

B(x)  -    lim  E[{x(t+At)-x(t)}  i  x(t)=x]?  (4g) 

At  ■+   0 

where  E[X]   denotes  the  expectation  of  the  random  variable  X. 
Furthermore,   a(x)   is  the  infinitesimal  variance  and  is  given  by 

a(x)  -   lim  V[{x(t+At)-x(t)}  |  x(t)=x]>  ^ 

At  ->  0 

where  V[X]   denotes  the  variance  of  the  random  variable  X.   (The 
definitions  (48)  and  (49)  involve  conditional  mean  and  conditional 
variance.)   The  process  corresponding  to  (47)  is  stationary,  since 
a  and   3  do  not  depend  upond  time.   The  specification  of  the 
infinitesimal  mean  and  variance  determines  the  Markov  process 
(uniquely) . 


7.2.   Diffusion  Approximation  to  the  Lanchester  Stochastic  Process. 

A  number  of  researchers  [45],  [29],  [15]  have  considered 
(in  a  heuristic  fashion  as  we  do  here)  the  limiting  behavior  of 
the  Kolmogorov  forward  equations  for  the  Lanchester  stochastic 
process  for  large  numbers  of  combatants.   However,  we  feel  that 
the  approximation  to  the  combat  attrition  random  process  that  each 
developed  was  inappropriate,  since  the  random  nature  of  the  attri- 
tion process  had  been  destroyed  in  the  passage  to  the  limit.   (There 
apparently  are  several  unanswered  theoretical  questions  in  this 
area,  especially  since  Etter's  development  [15]  is  very  precise. 
It  should  be  noted  that  Etter's  results  are  obtained  by  using  Lax 
and  Richtmyer's  well-known  theory  (a  key  aspect  being  Lax's  equiva- 
lence theorem  (see  p.  44  of  [36]))  on  the  convergence  of  finite 
difference  operators  to  differential  operators.   This  latter  theory 
was  developed  for  deterministic  problems  in  the  physical  sciences 
(hydrodynamics,  transport  theory,  etc.)  and  may  require  modificatic 
for  probabilistic  systems.) 

Before  presenting  our  new  results ,  let  us  show  the  development 
of  the  (inappropriate)  approximation  given  in  [45],  [29],  [15]. 
For  simplicity,  let  us  consider  the  Lanchester  square-law  attrition 
process  for  combat  between  two  homogeneous  forces.   The  Kolmogorov 
forward  equations  are 

for  0<m<M,  0<n<N 


1-43 


— (m,n,t)  =  a  n  P(m+l,n,t)  +  b  m  P(m,n+l,t)  -  (an+bm)  P(m,n,t),   (50) 


where  m  and  n  are  non-negative  integers.   Considering  large 
numbers  of  combatants,  it  does  not  seem  inappropriate  to  replace 
m  and  n  by  x  and  y  which  no  longer  have  to  be  integers. 
We  note  that  when  m  was  restricted  to  integer  values,   m  +  1  = 
m  +  Am  and  similarly  for  n.   Observe  that  here  we  have  Am  =  An 
It  is  convenient  to  write  (50)  as 


=  1. 


dP 

— (m,n,t)  =  an[P(m+l,n,t)  -  P(m,n,t)]  +  bm[P(m,n+l,t)  -  P(m,n,  t)  ]  .  (51) 


Now  when  we  replace  m  and  n  by  x  and  y,   we  may  think  of 
(51)  as 


_3P 

9t 


=  ay 


P(x+Ax,y,t)-P(x,y,t) 

Ax 


"  +  LArP(x,y+Ay,t)-P(x,y,t)l 


where  we  allow  fractional  casualties  in  our  formulation.   Letting 
Ax,  Ay  ■>  0,  we  (formally)  obtain 


ay  3x  +  bX  3y   3t     ' 


(52) 


with  initial  condition 


p(x,y,t=0)  =  6(x-xQ)  6(y-yQ), 


where 


p(x,y,t)  dx  dy  =   Prob 


at   t   the  X  force  level  is  between  x  and 

x+dx   and  the  Y   force  level  is  between  y   and  y+dy 


]. 
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i.e.   p(x,y,t)   is  a  probability  density  function. 

The  solution  to  (52)  is  readily  obtained  by  characteristics 
to  be 


dx   dy_   d_t   dp_ 
ay  "'  bx  ""  -1  ""  0  ' 


or 


p(x,y,t)  =   constant,  for 

dx 

dt"  =  ~  ay' 

f*  -  "  bx.  (53) 

We  observe  that  (53)  are  just  the  deterministic  Lanchester  equations 
corresponding  to  (50).   Using  the  initial  condition  p(x,y,t=0)  = 
<5  (x-xn)  6  (y-yn)  ,   it  is  easy  to  show  that  (53)  holds  for  the 
average  force  levels,  i.e. 

dx 

d7  =  ~  ay> 


bx,  (54) 


oo    oo 


where  x(t)  = 


dt 
x  p(x,y,t)  dx  dy,   etc..   We  observe  that 


—  OO   —00 


the  system  average  is  identical  with  the  sample  path.   Thus,  the 
limit  process  (52)  is  inappropriate  for  approximating  the  Lanchester 
stochastic  process  (50) ,  since  (52)  is  the  equation  for  a  deter- 
ministic process. 
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Thus,  we  seek  an  approximation  which  preserves  the  random 
nature  of  the  attrition  process.   Following  W.  Feller  (see  p.  235 
of  [17];  see  also  p.  235  of  [11]  and  [23]),  we  will  construct  an 
approximation  to  a  stochastic  process  with  discrete  states  by 
considering  a  corresponding  Markov  process  with  continuous  state 
space,  which  has  an  equivalent  infinitesimal  mean  and  variance. 

We  begin  by  considering  a  battle  with  discrete  time  steps. 

Over  each  time  step,  the  number  of  casualties  that  a  Y   soldier 

2 
produces  is  a  random  variable  with  mean  y   and  variance  a 
r  a  a 

and  similarly  for  X.   We  let 


X  =  X-f orce  level  at  time  step  n   (a  random  variable) , 
n 


Y   =  Y-force  level  at  time  step   n   (a  random  variable) . 
n 


Then 


E[X  ,  -X  X  =x,Y  =y]  =  -  yy  , 
n+1  n '  n    n  a 


and 


.  2 

V[X  ..-X  X  =x,Y  =y]  =  ya   . 

n+1  n '  n    n        a 

This  suggests  approximating  the  infinitesimal  means  and  variance 
as  follows : 


=  -  any   and   a  =  Ay, 
X      1         X 
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b  x  and  ay  =  Bx,  (55) 


where  a  ,  b  ,  A,  B  >  0.   Then  the  equation  for  p(x,y,t)   would 
be 


Ay 


^-+Bx^ 


2        2 
3x       3y  J 


+  V^+bl*tM^'       "6) 


with  initial  condition  p(x,y,t=0)  =  S(x-x„)  S(y-y_). 

We  observe  that  as  long  as   A  >  0   and   B  >  0   the  probability 
density  diffuses  over  time.   For  A  =  0   and   B  =  0,   we  have  a 
deterministic  process.   Equation  (56)  is  the  diffusion  approxi- 
mation to  the  Lanchester  (square-law)  stochastic  process  and  is 
the  major  result  of  this  section.   Unfortunately,  (56)  is  not 
readily  solved  by  the  method  of  characteristics  as  was  the  first 
order  partial  differential  equation  (52) . 

7.3.   Diffusion  Approximation  to  Fundamental  Functional  Equation. 

Based  on  our  consideration  of  diffusion  approximations  in 
Sections  7.1  and  7.2  above,  a  diffusion  approximation  to  the 
fundamental  functional  equation  (15)  is  easily  written  down. 
Recalling  that   t  =  T  -  t   is  "backwards  time"  and  considering 
the  retrospective  (backward)  evolution  of  the  system  from  end 
of  battle   t  =  T  (or  equivalently   t  =  0) ,  the  appropriate 
approximation  is  given  by 
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|f  Ct,VV7)  -  -  (blXl+b2x2)  ||+  -i-  (BlXl+B2x2)  ^ 

9y 


+  maximum 
0  £  <}>  <: 


JM-^^K) 


3x1  3x2 

with  terminal  condition 

S(T=0,X;L,x2,y)  =  ry(T)  -  px;L(T)  -  qx2(T). 

In  (57)  we  have  the  backward  operator  (using  "backward  time")  for 
the  continuous  Markov  process  (see  [20]). 

8.   Proposed  Future  Research. 

The  results  presented  in  this  appendix  are  preliminary  and 
should  be  expanded  in  two  directions:   (1)  computational  and 
(2)  theoretical.   We  propose  both  to  ONR  as  future  research  tasks, 
with  the  computational  investigations  deserving  the  higher  priority. 

The  most  important  future  task  is  to  generate  some  numerical 
results  via  the  finite  difference  approximation  (33) .   As  noted 
above,  the  adequacy  of  the  finite  difference  approximation  (in 
particular,  the  time  step  size)  can  be  checked  by  considering  the 
(partial)  exact  analytic  solution  that  has  been  developed.   A 
M.  S.  thesis  student  (R.  Powers,  Lt.,  USN)  is  currently  performing 
this  computational  investigation.   (A  previous  investigation  by 
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S.  Silvasy,  Maj . ,  USA,  proved  to  be  inconclusive.)   Results  of 
computations  for  the  optimal  control  of  the  Lanchester  stochastic 
process  can  then  be  compared  with  results  on  the  optimal  control 
of  the  deterministic  Lanchester  process.   The  goal  of  this  pro- 
posed investigation  is  to  determine  the  effect  of  the  nature  of 
the  attrition  model  (deterministic  or  stochastic)  on  the  structure 
of  the  optimal  target  selection  policy.   Additionally,  it  is  of 
interest  to  know  if  the  optimal  policy  for  the  deterministic 
process  is  valid  for  small  numbers  of  combatants  and  a  stochastic 
attrition  mechanism. 

Theoretical  investiations  should  center  about  gaining  more 
insight  into  the  structure  of  the  optimal  target  selection  policy 
when  the  attrition  mechanism  is  stochastic.   Topics  such  as  steady- 
state  behavior  of  equation  (15)  ,  further  development  of  the  analytic 
solution,  development  of  an  analytic  solution  to  the  diffusion 
approximation  (57)  ,  etc.  merit  further  investigation.   Additionally, 
the  theoretical  justification  of  passage  to  the  limit  for  the 
birth  and  death  equations  should  be  investiated  further. 
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Appendix  J.   Inertial  Combat. 

1.   Introduction. 

In  all  the  target  selection  problems  that  we  have  considered 
previously  in  this  report  an  implicit  assumption  has  always  been 
that  fire  could  be  instantaneously  shifted  from  one  target  type  to 
another.   To  illustrate,  let  us  recall-  a  typical  problem: 


maximize   {ry(T)  -  pxn (T)  -  qx0(T)}, 

HO  x  2 

dx1 


subject   to:     jj—  =   -   fa  y, 
dx 

dt-=  -(1-*>  V 


d^  =  -  blXl  -  b2X2' 
x1,x2,y  ^  0  and  0  £  cj>  £  1.  (1) 

In  this  problem  cf>  is  the  control  (decision  or  policy)  variable. 
The  reader  should  note  that  although  the  control  must  satisfy  the 
condition  0  ^  $  ^  1>  the  rate  of  change  of  <f>  is  unrestricted  so 
that  4>  can  instantaneously  change,  for  example,  from  0  to  1. 
Physically,  this  means  that  we  are  assuming  that  the  Y-forces  can 
instantaneously  shift  fires  as  desired. 

When  one  considers  command  and  control  problems  in  combat,  the 
above  implicit  assumption  on   <J>   (instantaneous  jumps  permitted)  does 
not  seem  to  be  a  realistic  assumption.   A  better  assumption  appears 
to  be  that  there  is  a  limit  to  how  fast   <f>   can  be  changed.   We 
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have  thus  been  led  to  consider  target  selection  problems  in  which 
the  rate  of  change  of  the  allocation  variable  is  bounded  (i.e. 
instantaneous  shifts  in  fire  are  not  allowed) .   For  reasons  dis- 
cussed below,  we  have  chosen  to  call  such  a  situation  "inertial 
combat ." 

Although  problems  in  which  curves  are  restricted  to  lie  in  a 
given  domain  were  considered  in  the  classical  calculus  of  variations 
as  long  ago  as  1831  [3]  (see  also  [1])  and  discussed  by  Weierstrass 
in  his  lectures  of  1879  (see  p.  395  of  [1]),  development  of  opti- 
mality  conditions  for  optimal  control  problems  with  state  variable 
inequality  constraints  has  been  accomplished  only  comparatively 
recently.   We  have  discussed  such  problems  with  a  state  variable 
inequality  constraint  (SVIC)  in  Appendix  E.   Recent  activity  appar- 
ently owes  its  origin  to  the  work  of  Gamkrelidze  (for  an  English 
translation  of  his  original  work  see  Chapter  VI  of  [8]).   Gamkrelidze 
points  out  that  in  many  physical  problems  there  are  restrictions  not 
only  on  the  control  parameters  but  also  on  the  state  (phase)  space. 
He  (see  p.  263  of  [8])  refers  to  piecewise  continuous  controls  as 
"inertialess  controls,"  since  such  controls  can,  if  need  be,  instan- 
taneously jump  from  one  value  to  another.   Following  Gamkrelidze 
then,  we  use  the  term  inertial  combat  to  refer  to  a  target  selection 
problem  in  which  the  rate  of  change  of  the  distribution  of  fire  is 
bounded. 

The  problem  that  we  shall  study  requires  more  than  the  theory 
of  Gamkrelidze  (see  Appendix  E  for  further  discussion  and  references) 
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which,  of  course,  is  for  optimal  control  problems  involving  in- 
equality constraints  on  a  function  of  the  state  variables  with  no 
explicit  dependence  on  the  control  variables.   We  recall  that  we 
say  that  the  problem  has  a  k —  order  SVIC  when  the  first  time 
(total)  derivative  of  the  state-variable  constraint  which  explicitly 
contains  the  control  variables  is  the  k — .   The  problem  that  we 
shall  study  has  a  second  order  SVIC.   Unfortunately,  there  appar- 
ently has  not  been  developed  any  condition  analogous  to,  using  the 
notation  of  Appendix  E,   y  £  0   for  a  k —  order  SVIC   (k  >  1). 
Using  the  method  of  adjoining  the  state-variable  constraint  directly 
to  the  criterion  functional  with  an  additional  Lagrange  multiplier, 
we  will  develop  a  necessary  condition  of  optimality  on  a  constrained 
subarc  that  corresponds  to,  again  using  the  notation  of  Appendix  E, 
having  a  restriction  on   y.   This  latter  condition  apparently 
arises  because  of  the  problem's  special  structure  and  does  not 
appear  to  be  a  general  condition. 

The  method  of  directly  adjoining  the  state-variable  constraint 
was  apparently  first  considered  by  Chang  [2].   However,  there  are 
some  gaps  in  his  development.   A  more  careful  treatment  is  by 
Speyer  and  Bryson  [10].   Mclntyre  and  Paiewonsky  [7]  also  discuss 
the  method  of  direct  adjoining  of  the  SVIC  (and  its  relationship 
to  the  method  of  adjoining  the  appropriate  time  derivative  of  the 
SVIC) .   It  may  be  shown  that  results  obtained  by  the  method  of 
direct  adjoining  of  the  state-variable  constraint  correspond  to 
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those  obtained  by  direct  application  of  the  Kuhn-Tucker  conditions 
in  a  Banach  space  (i.e.  consider  a  finite  difference  approximation 
to  a  variational  problem  (a  finite-dimensional  optimization 
problem)  and  then  pass  to  the  limit) . 

The  results  presented  in  this  appendix  are  preliminary  in 
nature  and  tentative,  obtained  from  our  first  cursory  examination 
of  such  a  problem.   However,  we  feel  that  this  is  a  promising 
area  for  obtaining  results  and  insights  into  principles  of  target 
selection.   We  propose  to  ONR  further  examination  of  such  problems 
as  a  future  research  task. 

2 .   The  Optimal  Control  Problem. 

Accordingly,  we  consider  the  following  problem: 


maximize   {ry(T)  -  px  (T)  -  qx~(T)}  with  T  specified 
u(t) 

dxl 


subject  to:  ^— -  =  -  cf^y, 
dx 


'  =  -  (1  -  4>)a0y, 


dt      v    T/  2- 


f*  =  *b-A  -b2x2, 


of  "  U>  C2) 


:.,,  x  ,  y  ;>  0>   0  £  (j>  £  1,   and   -A^u^B, 
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where 

x..(t),  x2(t),  y(t)   are  force  levels, 

p,  q,  r  are  utilities  assigned  survivors, 

a^ ,  a„,  b  ,  b_   are  (constant)  attrition-rate  coefficients, 

<}>   is  the  fraction  of  Y   fire  directed  at  X  , 

and   u  is  the  rate  of  change  of  <t>. 

We  will  focus  primarily  on  the  development  of  the  basic  neces- 
sary conditions  of  optimality  for  (2).   For  this  goal,  of  course, 
the  nature  of  the  planning  horizon  (terminal  target  set  or  prescribed 
duration)  doesn't  make  any  difference. 

The  reader  should  note  that  the  control  variable  in  problem  (2) 
is  u,  while  <t>   (the  control  variable  in  problem  (1))  is  now  a 
state  variable.   Hence,  the  restriction  0  £  <|>  £  1  is  now  a  (first 
order)  SVIC.   When  we  use  the  approach  of  Gamkrelidze,  we  handle 
it  as  follows: 

for  $  -  1  £  1,   we  must  have 

4^-  =  u  £  0  when  <J>  -  1  =  0 ,  (3) 

dt 

and 

for     -  <J>  £   0,   we  must  have 

-  4^"  =  -  u  £   0     when     -  4>  =   0.  (4) 

dt 
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To  avoid  being  encumbered  by  too  many  symbols,  we  will  consider 
only  the  SVIC  x  2:  0 .   Clearly,  we  lose  no  generality  in  doing 
so.   In  the  notation  of  Appendix  E,  we  have  then 


C  (t,x)  =  -  x±  z   0,  (5) 

dC     dxl 


d2C 


dt     dt-=*aly>  (6) 

~  =  uaxy  -  (()a1(b1x1+b2x2)  .  (7) 


dt 

Now,  when  x  (t)  =  0   for   L  ^  t  i  t-  =  I,  we  control  the  system 
by  requiring 


C    (t,x)   =   -  xx  =   0,  (8) 

dC  dxl 

or  (t>x)  =  ~dT  =  *aiy  =  °>  (9) 


and 

2  =  uaxy   -  4)a1(b1x1+b2x2)  <;   0.  (10) 


d2C 


dt 

Hence,  the  origin  of  the  term  second  order  SVIC.   Clearly,  when 
X- (t)  =  0   for  a  finite  interval  of  time,  by  (9)  we  must  have 
4>*(t)  =  0   (since  y  >  0)   and  then  (10)  yields  u*Ct)  =  0, 
where  we  have  considered  the  state  equations  (2) . 

3 .   Necessary  Conditions  of  Optimality  on  Constrained  Subarc  for  $_• 

When  Gamkrelidze's  approach  (see  Chapter  VI  of  [8])  and  con- 
siders (3)  ,  (4)  ,  (8)  ,  (9)  ,  and  (10)  ,  then  the  Hamiltonian  is  given  by 
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H(t,   xi(p.,u)    =   -  p1<J>a1y  -  p2(l-<|>)a2y  -  p3(blXl+b2x2) 
+  P4u  -  n1(t)u  +  n2(t)u  -  y(t){uaiy  -  ^(b^+b^) } ,  (11) 


where 


n1(t) 


n2(t) 


=  0  for  4>  <  1, 

;>  0  for  4>  =  1, 

=  0  for  <J>  >  0, 

;>  0  for  <J>  =  0, 


and 


y(t) 


=  0  for  x  >  0, 
;>  0  for  x  =  0. 


We  have  adopted  above  the  following  correspondence  between  state  and 
dual  variables: 


state  variable 


dual  variable 


y 

4> 


'4* 


The  adjoint  system  of  differential  equations  for  the  dual  variables 
is 

dp, 


—  =  blP3-  y(tHalbl, 


(12) 
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dF 
dp, 
dF 

dp, 
dt 


b2p3   -   p(t)()»a1b1> 


*alpl  +    ^1_*)a2p2  +  ^^aiu> 


(a^   -   a2p2)y   -  u  (Oa^b^+b^) 


(13) 
(14) 
(15) 


Assuming  that  time  determines  the  termination  of  the  battle  (planning 
horizon),  i.e.  we  don't  consider  termination  due  to  annihilation 
of  one  side  or  the  other  before   t  =  T,   the  boundary  conditions 
for  the  dual  variable  at   t  =  T   are  given  by 


P1(t=T)  = •-  p  +  vx, 


where 


=  0   for  xx(T)  >  0, 
;>  0   for  x2(T)  =  0, 


(16) 


P2(t=T)  =  -  q  +  v2, 


where 


=  0   for  x  (T)  >  0, 
w  :►  0   for  x2(T)  =  0, 


(17) 


P3(t=T)  =  r  +  v3 


where 


=  0   for  y(T)  >  0, 
£0   for  y(T)  =  0, 


(18) 


P4(t=T)  =  v4  "  v5 


where 
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and 


=  0  for  <J>(T)  >  0, 

;>  0  for  <J)(T)  =  0, 

=  0  for  <f)(T)  <  1, 

;>  0  for  4>CT)  =  1. 


(19) 


When  x  ,  x  ,  y  >  0   and   0  <  <J>  <  1,   the  control  law  is  deter- 
mined by  Pontryagin's  maximum  principle.   Hence,  we  consider 


maximize  H(t ,x. ,p . ,u) , 
-  A  £  u  £  B    1   1 


and  this  yields 


u*(t)  = 


B  for  p,(t)  >  0, 

„  -A  for  p. (t)  <  0. 
4 


(20) 


We  must  further  investigate  the  possibility  of  a  singular  subarc 

3H 
[5],  [6]  on  which  —  =  0   for  a  finite  interval  of  time  (so  that 

oU 

3H 
all  time  derivatives  vanish) .   The  condition  that  —  =  0  yields 

3u 

that  on  a  singular  subarc  we  must  have 


p4(t)  =  0. 


(21) 


The  condition 


dt 


3H 
3u 


0   then  yields 


alPl(t)  =  a2P2(t) 


(22) 
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Proceeding  to  the  next  time  derivative,  we  would  have  on  the  singular 
subarc  when  (22)  holds  that 

,2 


at2 


8H 


3u 


=  yp3(a1b1  -  a2b2).  (23) 


Considering  (23),  we  see  that  a  singular  solution  is  impossible. 
On  a  constrained  subarc  on  which  cj>(t)  =  1   for   t-  £  t  £  t. 

the  control  is  determined  by  -~  =   0   and  hence 

u*(t)  =  0   for   t   <  t  <  t  .  (24) 

The  multiplier   n, (t)   is  determined  by  the  condition  —  =  0  and 
r       1  J  .      3u 

hence 


nx(t)  =  p4(t).  (25) 

The  condition  that   n  (t)  ^  0  yields  that  on  the  constrained  subarc 
we  must  have 


p4(t)  £  0.  (26) 

Differentiating  (25)  and  combining  with  (15)  ,  we  find  that 


n1(t)  =  yU^  -  a2p2) ,  (27) 

so  that  Gamkrelidze's  condition  n  (t)  ^  0   [8]  is  only  satisfied 
on  the  constrained  subarc  with  $   =   1  when 


a1(-p1(t))  ;>  a2(-p2(t)),  (28) 


J-ll 


which  the  reader  will,  of  course,  recognize  as  a  result  for  the 
corresponding  "inertialess"  combat  problem.   Denoting  the  time  of 
an  entrance  corner  by   t1   and  that  of  an  exit  corner  by   t_, 
the  corner  conditions  (see  Appendix  E  for  discussion)  yield  that 
at  an  entrance  corner 


and 


or 


Pi(t1-)  =  PiCt^)   for  i  =  1,2,3,  (29) 


P4(t1-)  =  0  =  P4(t1+)  -  n1(t1+),  (30) 


P4(tx+)  -  n1(t1+),  (3D 

where   t--  denotes  a  left-hand  limit.   The  reader  should  note  that 
(31)  is  in  consonance  with  (25).   Futhermore,  at  an  exit  corner  we 
have 

p.(t2-)  =  p±(t2+)   for   i  =  1,2,3,4.  (32) 

Considering  either  luCt^-)  =  0  or  H(t2~)  -  H(t2+)  ,   we  readily  find 

that 

P4(t2-)  =  0  =  P4(t2+).  (33) 

Considering  (27),  (31),  and  again  n-Ct^-)  =  0,  we  see  that  when 
there  is  an  exit  from  the  constrained  subarc  at   t  =  t,,,   then 
(31)  becomes 

* 

P4<tj+)  -  n1(t1+)  =    y(a2P2  "  aipi}  dt  *  °*      (34) 

t, 
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On  a  constrained  subarc  on  which  <$>(t)    =  0   for   t-  £  t  si  ti- 
the control  is  determined  by  -~-   =  0   and  hence 

dt 

u*(t)  =0   for   t1  <  t  <  t2.  (35) 

The  multiplier  r)~(t)   is  determined  by  the  condition  -r—  =  0 
2  3u 

and  hence 


n2(t)  =  -  p4(t).  (36) 

The  condition  that   n~(t)  ^  0  yields  that  on  the  constrained  subarc 
we  must  have 


P4(t)  £  0.  (37) 


Differentiating  (36)  and  using  (15) ,  we  find  that 


ii2(t)  =  y(a2p2  -  a1p1)  ,  (38) 


so  that  Gamkrelidze's  condition  n9(t)  ^0   is  only  satisfied  on 
the  constrained  subarc  with  <±>  =  0  when 


ai(-Pl(t))  <:  a2(-p2(t)),  (39) 

which  again  the  reader  will  recognize  as  a  result  which  also  arose 
in  the  corresponding  "inertialess"  combat  problem.  At  an  entrance 
corner,  the  corner  conditions  yield 


Pi(t1-)  =  P^t^)   for  i  =  1,2,3,  (A0) 

and 
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P4(t1-)  =  o  =  p4(tx+)  +  n2Cta+>,  (41) 

or 

P4(t1+)  =  -  n2(t1+).  (42) 

Again,  we  point  out  to  the  reader  that  (42)  is  in  consonance  with 
(36).   Furthermore,  at  an  exit  corner  we  have 


P1(t2-)  =  Pi(t2+)   for  i  =  1,2,3,4.  (43) 

Considering  either  n2(t2~)  =  0  or  H(t2~)  =  H(t  +) ,  we  readily 
find  that 


p4(t2-)  =  0  =  P4(t2+).  (44) 

Considering  (38),  (42),  and  again  n2(t2~)  =0,  we  see  that  when 
there  is  an  exit  from  the  constrained  subarc  at  t  =  t„ ,   then 


(42)  becomes 


P4(tr)  -  -  n2(t  +)  = 


2 


h 


y(a2p2  -  a1p1)dt  £  0.        (45) 


4 .   Partial  Synthesis  of  Optimal  Policy  When  x,  ,  x  >_  £. 

In  this  section,  we  will  give  some  preliminary  results  on  how 
the  necessary  conditions  of  the  last  section  may  be  used  to  determine 
the  optimal  policy.   We  recall  that  we  developed  those  results  for 
au,  x2  >  0. 
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Since  we  develop  the  solution  to  this  problem  by  working  back- 
wards from  the  end   t  =  T,   it  is  convenient  to  introduce  the 

"backwards  time"  variable   x  defined  by   x  =  T  -  t.   We  observe 

2  2 

d      d   u    d  d     T   . 

that  —  =  -  3—  but  — _  =  — _.   It  is  also  convenient  to  define 

dt     dx         2  ,  2 

dt  dx 


v(t)  =  a1p1(t)  -  a2p2(t),  (46) 

so  that  differentiating  (46)  and  combining  with  (12)  and  (13) ,  we 
find  that 

dt-»3<t)('lbl-'"2b2i-  (47) 

Let  us  now  assume  (without  any  loss  of  generality)  that  _§.-,]!■,  >.  a.„b_  . 
It  is  easy  to  show  that  p~(t)  >  0   for  all   t  and  hence 

^T   (t)  >  0   for  all   t.  (48) 

at 

Using  (46)  ,  it  is  convenient  to  write  (15)  (for  x..  >   0)   as 


dP4 

=  yv,  (49) 


dt 

and  hence 

d2 

fr =  -  (W^V  v  +  *  fr  (50) 

dt 
In  synthesizing  an  optimal  trajectory  there  are  two  cases  to  be 
considered: 


Case  (a)     a  p  ^  a,,q  , 
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Case  (b)    a  p  <  a  q. 

For  Case  (a):   a.p  s  a_q ,   it  is  convenient  to  first  observe 

that  a  Taylor  series  expansion  of  p,  (t)   yields 

2 
dp  2  d  p, 

P4(t)  =  p4(x=0)  +  t  j^-   (t=0)  +  \ y   (t=t)>         (51> 

dx 

where   t £  (0,t) . 


In  this  case  we  have 

v(x=0)  =  -  alP  +  a2q  £  0,  (52) 

so  that  considering  (48)  it  is  readily  seen  that 

v(x)  <  0  for  x  >  0,  (53) 

and  hence  (49)  and  (50)  readily  yield 

dp, 

-^-   (x=0)  =  y  (aLp  -  a2q)  ;>  0,  (54) 

,2 
d  P4  (t)  >  0.  (55) 

dx2 
Now,  there  are  three  subcases  to  be  considered  when  a^p  2:  a,,q ; 

Subcase  (al)   <Kt=T)  =  0, 

Subcase  (a2)    0  <  4>(t=T)  >  1, 

Subcase  (a3)    4>(t=T)  =  1. 
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We  shall  now  show  that  subcase   (al)   is  inconsistent  with  an 
optimal  policy  and  work  out  the  solution  for  the  other  two  cases 


Subcase 


(al)    <Kt=T)  =  0  when   a  p  ^  a  q. 


Since   <}>(t=T)  =  0,   (19)  yields  that 


p^(T=0)  =  v4  ;>  0.  (56) 


Then  (51),  (54),  (55),  and  (56)  yield  that 

P4(x)  ;>  0   for   x  ;>  0,  (57) 

with  strict  inequality  holding,  i.e.   P/Ct)  >  0,   for   x  >  0.   If 
we  were  on  a  constrained  subarc  for  a  finite  interval  of  time,  i.e. 
<f>(t)  =  0   for   t,  2:  t  ^  T,   then  (57)  is  inconsistent  with  (37) 
(also  note  that   ru(t=T)  =  y(a-,P  -  a?q)  ^  0,   a  violation  of 
Gamkrelidze's  condition   n  (t)  ^  0   (when  a  p  >  a„q)).   If  we 
were  not  on  a  constrained  subarc  for  a  finite  interval  of  time, 
we  again  reach  a  contradiction.   To  see  this,  let  us  observe  that 
(20)  and  (57)  yield  that 

u*(x)  =  B   for   x  ^  0,  (58) 

so  that  a  backward  integration  of  the  state  equation  (2)  yields  that 

<Kt)  =  -  Bx, 

which  is  impossible.   Hence  this  case  is  inconsistent  with  an  optimal 
policy. 
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Subcase  (a2)   0  <  4>(t=T)  <  1  when  a  p  ;>  a  q. 

Since  0  <  <J>(t=T)  <  1,   (19)  yields  p4(t=0)  =  0,   and  (51), 
(54),  and  (55)  again  yield  (57).   Hence,  (58)  again  holds.   Denoting 
4>(t=0)   by  <(>0,   it  is  easily  seen  that 


<|>n  +  BT  =  <|>(t«T)  <  1, 


(59) 


so  that  this  case  happens  when 


T  < 


l-*0 


,  B 


(60) 


The  optimal  policy  is  then  given  by 


u*(t)  =  B   for  0  £  t  £  T  < 


l-4>, 


(61) 


For  larger  time  T,  we  must  go  to  the  next  subcase 


Subcase  (a3)    4> (t=T)  =  1  when  a..p  ^  a?q. 


Since  4)(t=T)  =  1,   (19)  now  yields 


P4(t=0)  =  -  v5  ^  o. 


(62) 


It  may  be  shown  that  we  get  a  contradiction  unless  v,  =  0  so 
that  we  must  have 


p4(x=0)  =  0, 


(63) 
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If   <{>(t)  <  1   for  T  -  6  £  t  <  T  where  6  >  0,   then  the  devel- 
opment of  the  previous  subcase  holds.   If  we  are  on  a  constrained 
subarc  for   t  ^  t  ^  T,   then  (recalling  (46))  (53)  is  equivalent 
to  satisfying  (28)  (Gamkrelidze  's  condition  ii  ^  0).   Hence,  we 
can  remain  (in  our  backwards  progression  from  the  end   t  =  T)   on 
the  constrained  subarc  until  we  have  to  get  off  to  meet  the  initial 
condition  <}>(t=0)  =  4>n*   ^s  we  wor^  backwards  and  leave  the  con- 
strained subarc  (i.e.  in  forwards  time,  enter  at   t  ),   the  corner 
condition  (30)  yields  that  for   0  £  t  £  t. 


dp  (t  -t)2  d2p 

p4(t)  =  -(trt)  ^  (t=ti)  +_1__    4  (t=t)j 

dt 


(64) 


where  t  €  (t,^)  . 


Recalling  (54)  and  (55),  we  see  that  P,(t)  ;>  0   for  0  £  t  £  t 

so  that  the  rest  of  the  analysis  is  similar  to  the  preceeding 
subcase.   Hence, 

'l-( 


for  T  ^ 


0 


B 


we  have 


u*(t)  = 


0   for 


3  £    t 

<. 

I 

^                    J 

<    t  £   T, 

I  B     J 

(65) 
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<|>*(t)  = 


<j>0  +  Bt  for  0  £  t  £ 


l-<\ 


for 


l-< 


<  t  £  T, 


(66) 


For  Case  (b) :   a  p  <  a„q,   we  now  have  that  (recalling  (46)) 


v(x=0)  =  -  a  p  +  a2q  ^  0. 


(67) 


Recalling  (48)  ,  we  see  that  at  some  (backwards)  time  v(x)   must 
become  zero.   Let  us  denote  this  "backwards  time"  as  x  .   Thus, 


v(t=t  )  =  0, 


(68) 


Again,  there  are  three  subcases  to  be  considered  when  a..p  <  a«q 


Subcase  (bl)   <J>(t=T)  =  0, 


Subcase  (b2)   0  <  <f>(t=T)  <  1, 


Subcase  (b3)    <{>(t=T)  =  1. 


Analysis  of  these  subcases  is  similar  to  that  given  for  Case  (a) , 
with  Subcase  (b3)  being  impossible. 

Let  us  now  observe  (recalling  (28)  and  (39))  that  in  order  to 
satisfy  Gamkrelidze's  condition  on  the  rate  of  change  on  a  Lagrange 
multiplier,  we  must  have 
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v(t)  ^  0   when  <J>(t)  ■  1   for  a  finite  interval  of  time,     (69) 
and 

v(t)  ^  0   when   4>(t)  =  0   f°r  a  finite  interval  of  time.     (70) 

We  have  previously  noted  the  correspondence  of  these  results  to  those 
for  "inertialess"  combat.   We  now  consider  the  case  when  4>(t=T)  =  0. 
Let   t    denote  the  (forward)  time  when  we  enter  a  constrained 
subarc  with  <j>  =  0 ;   similarly   t~    the  time  of  leaving  one  with 
<J>  =  1.   We  further  assume  that 

<j>(t)  =  0   for   t  °  £  ti  T.  (71) 

We  now  prove  that  it  is  impossible  to  have  v(t=t_  )  =  0, 
i.e.  v(t)  must  be  >  0  before  cf>  =  0.  The  corner  condition 
(41)  and  (49)  yield  that  for   t  £  t  ° 

2   2 
(t  -t)Z  dZp 

pA(t)  »  —^ r-  (t=t) ,  (72) 

*  dt 


where   t  €  (t,t  ) , 

so  that  (recalling  that   v(t=t   )  =  0,   (48),  and  (50))  it  is  readily 
seen  that 


p.  (t)  >  0   for   t  <  t.°.  (73) 

4  1 
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By  (20)  this  implies  u*(t)  =  B  for   t  <  t  °,  which  clearly  is 
impossible. 

Now,  it  is  readily  shown  that  for  t_  £  t  sS  t1 


P4(t)  =  - 


y(t)v(t)dt. 


(74) 


In  particular 


P4(t2   )  =  - 


y(t)v(t)dt. 


(75) 


Considering  the  corner  condition  (33)  and  (48) ,  it  is  clear  that 


v(t=t2  )  <  0, 


(76) 


while 


v(t=tp  >  0, 


(77) 


We  may  also  write 


PA(t)  = 


y(t)v(t)dt. 


(78) 


Considering  (76)  through  (78),  it  should  be  clear  that  p^(t)  <  0 


for   t9   <  t  <  t    and  hence  by  (20) 


1         o 
u*(t)  =  -  A  for   t2  <  t  <  tx  . 


(79) 


Integration  of  the  state  equation  (2)  then  yields 
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<j>(t)    =   1   "  ACt-t^)      for      t*  ^    t  £    tl°, 


(80) 


and  hence 


o  11 

fcl  t2      '"  A" 


(81) 


The  times   t«   and   t..    are  determined  by  the  conditions 


T  "  Tl  ^  (t2  'll0) 


(82) 


and 


y(t)v(t)dt  =  0, 


(83) 


or 


T-t. 


y(t)v(t)dt  =  -     y(t)v(t)dt. 

T-t, 


(84) 


The  relationship  of  the  times   t_  ,  T  -  t  ,   and   t    to  the 
time  history  of   <J>(t)   is  shown  in  Figure  1. 
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4>=0 


& 


t ime ,  t 


Figure  1.   Relationship  of   t-  ,  T  -  t  ,   and 
t1    to  Values  of  <f>. 


Omitting  further  details  for  now,  we  do  reach  an  important  con- 
clusion:  for  "inertial"  combat  (see  (2))  one  begins  to  shift  fire 
earlier  in  forward  time  (anticipating  changes  in  target  priority) 
than  in  the  corresponding  "inertialess"  case.   Again,  the  reader 
is  referred  to  Figure  1  for  motivation  of  this  statement. 


5 .   Necessary  Conditions  of  Optimality  on  Constrained  Subarc  for  x. 

Let  us  first  consider  Gamkrelidze's  approach  of  considering 
the  time  derivative  of  the  state-variable  constraint  on  a  con- 
strained subarc  on  which  x, (t)  =0   (and  x~  >  0)   for   t  ^  t  ^  t„. 
We  limit  our  discussion  here  to  the  necessary  conditions  for  it 
to  be  optimal  to  derive  x,   to  zero.   Recalling  (8)  through  (10), 
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the  control  is  clearly  u*(t)  =  0.  It  should  be  noted  that  in  this 
case  we  have  a  second  order  SVIC.  Considering  (2) ,  we  must  clearly 
also  have  4>(t)  =  0.   However,  since  the  choice  of  control  is 

restricted  by  (10)  and  not  (4),  we  may  take   n9(t)  =  0.   The 

3H 
multiplier   u(t)   is  determined  by  the  condition  ■*—  =  0  and 

dU 

hence 

y(t)  =  — .  (85) 

ay 

The  condition  y(t)  ^  0  yields  that  on  a  constrained  subarc  we 
mus  t  have 


P4(t)  ;>  0,  (86) 

which  should  be  contrasted  to  the  necessary  condition  for  constrained 
subarc  with  <\>  =  0  and  x  >  0.  It  is  also  convenient  to  write  (85) 
as 


y(t)a]y  =  P4(t).  (87) 

Differentiating  (87)  with  respect  to  time  and  considering  (2)  and 
(15) ,  we  obtain 

ii(t)  =  ]"(alPl-a2P2)  .  (88) 

It  should  at  this  point  be  noted  that  Gamkrelidzers  condition 

y  ^  0  does  not  apparently  hold  for  higher  order  SVIC.   A  further 

differentiation  (and  use  of  (12)  and  (13))  yields 
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P3(t) 
Ut)   =~ (a1Va2b2>»  (89) 

where  we  have  used  the  fact  that  <t>(t)  =  0. 

We  now  will  show  that  apparently  due  to  this  problem's  special 
structure  a  necessary  condition  of  optimality  on  a  constrained 
subarc  with  x  (t)  =  0  is  that 

ti(t)  ;>  0.  (90) 

We  do  this  by  considering  the  method  of  adjoining  the  state-variable 
constraint  directly  to  the  criterion  functional  with  an  additional 
Lagrange  multiplier  [2],  [10].  When  we  add  the  state-variable 
constraint  directly,  the  Hamiltonian  is  given  by  [10] 

H(t,xi,pi,u)  =  -  p^a^  -  P2U-4>)  a2y  -  p^b^+b^)  +  p4u  +  yxr(91) 
where 


y(t) 


=0  for  x  >  0, 
k  ;>  0  for  x1   =  0, 


and  p.   and   y   denote  the  Lagrange  multiplier  when  this  method 
is  used  (as  opposed  to  p   and   y  which  denote  the  multipliers 
when  Gamkrelidze 's  approach  is  used).   The  condition  on  the  sign  of 
y(t)   when  x1  =  0  has  been  given  by  Mclntyre  and  Paiewonsky  [7] 
(see  also  [4]).   (It  is  readily  obtained  using  Valentine's  method 
by  considering  the  Legendre-Clebsch  condition  (see  [11]  for  the 
corresponding  development  in  mathematical  programming).)   The 
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adjoint  system  of  differential  equations  for  the  dual  variables 


is  now 


dp. 


=  blP_  -  y(t), 


dt    "lr3 


(92) 


dp2 

dt~  =  b2P3' 


(93) 


dp3 

dt"  =  ♦Vl  +  (1~^    a2P2 


(94) 


dp4 

irm  (aiPra2p2)y- 


(95) 


On  a  constrained  subarc  on  which  x..  (t) 


0   (and  x  >  0) 


for   t.  ^  t  ^  t9,   we  have 


3u   dt 


W 


3u 


=  0 


(96) 


and  the  multiplier   y(t)   is  determined  by  (96)  and  the  condition 


dt' 


8H 


9u 


=  0, 


(97) 


We  find  accordingly 


P,(t) 
(t)  --3— <^b v-a2b2). 


(98) 


The  condition  that   u(t)  ^  0  when  x  =  0   for  a  finite  interval 
of  time  yields  that 


albl*  a2b2' 


(99) 
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since  it  is  readily  shown  that  p^(t)  >  0.   Thus,  we  have  developed 
the  same  necessary  condition  for  it  to  be  optimal  to  drive  x 
to  zero  as  for  "inertialess"  combat, 

A 

Let  us  now  relate  the  above  result  for  y   to  y".   Considering 
(89)  and  (98) ,  we  have 

«&  -  m .  doo) 

P3(t)   p3(t) 

Assuming  that  y(t)  >  0,   consideration  of  the  boundary  condition 

a 

for  p  (t)   and  p~(t)   (18) ,  the  appropriate  adjoint  equation 

A  A 

(and  results  for  p, ,  p~,  p, ,   and  p„) ,   and  the  corner  conditions 
that 


P3(tr)  =  p3(t1+),  (101) 


and  the  condition  [10] 


P3(tr)  =  P3(t1+),  (102) 

A 

we  see  that  p~(t)   and  p~(t)   have  the  same  sign.   By  (100)  so 
do   y(t)   and   y"(t)  .   Hence,  we  have  shown  that 

y(t)  ;>  0  when  x^t)  =0  for  ^  £  t  £  t^  (103) 

6.   The  Special  Case  of  Unbounded  Rates. 

In  our  target  selection  problem  (2),  the  rate  of  change  of  the 
fraction  of  Y   fire  directed  at  X   (i.e.  -£)      is  restricted. 
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Thus,  we  must  have  for  u  =  -tJ- 


-  A  £  u  £  B.  (104) 

Let  us  observe  that  for  -  A  =  -  °°  and  B  =  +  °°  problem  (2) 
reduces  to  problem  (1) ,  the  "inertialess"  combat  problem.   Accord- 
ingly, we  would  expect  much  to  be  similar  in  the  solutions  to  these 
two  problems.   Indeed,  this  has  been  a  result  of  our  analysis:   in 
both  problems,  for  example,  a  necessary  condition  for  it  to  be 
optimal  to  drive  x..   to  zero  is  that  a->b     ^  a?^9' 

It  consequently  is  of  interest  to  develop  our  previous  results 
for  (1)  by  considering  (2)  with  u  =  -j-*-  unbounded.   In  this  case, 
we  must  allow  jumps  the  state  variable   <j> .   Such  problems  have  only 
recently  been  studied  in  the  literature  [9],  [12],  [13].   We  feel 
that  the  relationships  between  optimal  target  selection  policies 
for  "inertialess"  and  "inertial"  combat  should  be  further  examined. 

7.   Discussion. 


The  results  presented  in  this  appendix  are  preliminary,  being 
based  on  an  initial  cursory  examination.   We  feel  that  this  is  an 
important  and  promising  area  and  propose  this  to  ONR  as  a  future 
research  task. 

We  saw  that  for  both  "inertialess"  and  "inertial"  combat,  we 
must  necessarily  have 


albl  ^  a2b2' 
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in  order  for  it  to  be  optimal  to  drive  x..   to  zero  (while  x„  >  0 
and  before   t  =  T) .   Furthermore,  we  also  developed  a  necessary 
condition  involving  v(t)  =  a  p  (t)  -  a^p-Ct)   for  it  to  be 
optimal  to  have   <Kt)  =  0  or   1   for  a  finite  interval  of  time. 
Again,  the  results  were  similar  to  those  for  the  "inertialess" 
combat  problem.   A  significant  result  (see  Section  4),  however, 
was  that  for  "inertial"  combat  an  optimal  policy  for  the  distribu- 
tion of  fire  over  enemy  target  types  is  characterized  by  beginning 
to  shift  fire  before  target  priorities  (as  measured  by  the  sign 
of  v(t))   change. 
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Appendix  K.  Target  Selection  for  Helmbold's  General 
Attrition  Structure. 

1.   Introduction. 

One  of  the  objectives  of  this  research  sponsored  by  ONR  is  to 
investigate  the  sensitivity  of  the  optimal  target  selection  policy  to 
the  form  of  the  combat  attrition  model.  We  have  initiated  in  Appendix 
I  an  investigation  as  to  whether  for  the  simplest  target  selection 
problem  the  optimal  policy  is  sensitive  to  whether  attrition  is 
modelled  as  a  deterministic  or  a  stochastic  process. 

Let  us  now  restrict  our  attention  here  to  deterministic  combat 
formulations.  As  discussed  in  Appendix  H,  various  forms  of  Lanchester- 
type  equations  have  been  hypothesized  to  describe  combat  between  two 
homogeneous  forces.  These  formulations  are  readily  extended  to  combat 
situations  between  heterogeneous  forces.   In  Appendices  A,  B,  C,  E,  F, 
G,  and  J  we  have  considered  target  selection  problems  in  which  the 
rate  of  attrition  of  an  enemy  target  type  is  directly  proportional 
to  the  number  of  friendly  firers.   For  this  type  of  target-type  attri- 
tion we  saw  that  the  optimal  policy  was  always  to  concentrate  all  fire 
on  a  single  target  type.   The  index  of  the  target  type  upon  which  all 
fire  is  concentrated  may  change  (without  the  target  type  being 
annihilated)  over  the  course  of  battle,  but  we  always  concentrate  all 
our  fire.   In  Appendix  B  we  heuristically  discussed  why  this  is  so  in 
an  intuitive  fashion.   In  Appendices  B  and  D  we  consider  a  target 
selection  problem  in  which  the  rate  of  attrition  of  an  enemy  target 
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type  is  directly  proportional  to  the  product  of  the  number  of  firers 
and  the  number  of  targets.   For  this  type  of  target  type  attrition  we 
saw  that  the  optimal  policy  might  be  other  than  concentrating  all 
fire  on  a  single  target  type. 

We  were  thus  led  to  consider  a  general  form  of  (deterministic) 
combat  attrition  in  a  simple  target  selection  problem  in  order  to 
study  the  sensitivity  of  the  optimal  policy.   Specifically,  we  were 
interested  in  whether  the  optimal  policy  would  be  to  always  concen- 
trate all  fire  on  a  single  target  type.   In  1965  R.  Helmbold  proposed 
a  modification  of  Lanchester's  square  law  attrition  model  to  incorpo- 
rate inefficiencies  of  scale  for  the  larger  force  when  the  force  sizes 
are  grossly  unequal  [3].  Although  unfortunately  not  stated  in  this 
paper,  we  conjecture  that  this  suggested  modification  was  firmly 
based  upon  empirical  evidence:   Helmbold  noted  in  [1]  (see  also  [2]) 
that  if  one  analyzed  a  large  number  of  land  battles  as  if  each  were 
combat  between  two  homogeneous  forces,  then  the  ratio  of  attrition - 
rate  coefficients  was  inversely  proportional  to  the  initial  force 
ratio. 

Hence,  we  decided  to  study  the  two-against-one  target-type 
selection  problem  with  combat  attrition  following  Helmbold 's  general 
model.   In  this  appendix  we  present  preliminary  results  of  our  rather 
brief  examination.  We  shall  only  develop  the  basic  necessary  condi- 
tions of  optimality  (and  not  address  here  the  synthesis  of  optimal 
trajectories) .   These  preliminary  results  indicate  that  when  target-type 
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attrition  follows  Helmbold's  general  model,  the  optimal  policy  is 
always  to  concentrate  all  fire  on  a  single  target  type.   In  other  words, 
this  attrition  model  apparently  does  not  lead  to  a  singular  solution 
in  the  optimal  control  problem. 

2.  Helmbold's  Modification  of  Lanchester's  Equations. 

In  1965  Helmbold  [3]  proposed  a  general  Lanchester-type  model 
in  which  the  effectiveness  of  a  force  is  dependent  upon  the  force  ratio. 
A  special  case  (when  each  attrition  rate  is  proportional  to  a  product 
of  terms  one  of  which  is  a  power  of  the  force  ratio)  of  Helmbold's 
general  formulation  is 


dx       c  1-c 
dt 


tz  -  -  ax  y 


f*  =  "  by0*1"*,  (1) 


with  initial  conditions 


x(t=0)  =  xQ, 

y(t=0)  =  yQ, 

where  a  and  b  are  Lanchester  attrition-rate  coefficients,  x(t) 
and  y(t)   are  force  levels,  and  c  is  a  parameter  which  takes  on 
values  between  zero  and  one. 

Helmbold  gave  the  time  solution  to  (1)  as  (for  c  f4  1) 


:(t)  =  (xq1  °  cosh(l-c)/ab"  t  -  yQ1  C/^  sinh(l-c)  /ab"  t}1  °, 
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and 

1 


y(t)  =  {y  X  C  cosh(l-c)/ab~  t  -  x1  C/-   sinh(l-c)/ab"  t}1  C    (2) 

He  also  stated  that  equations  (1)  also  yield  that  state  equation 

,,   2-2c  2-2c.     ,   2-2c  2-2cN 

b(xQ    -x    )  =  a(yQ    -y    ),  (3) 

which  yields  for  various  values  of  the  parameter  c 


c  =  0  square  law. 

c  =  1/2  linear  law, 

c  =  1  "logarithmic"  law. 

When  c  =  1,   (3)  takes  the  form 

b(£n  x  -In   x)  =  a(£n  yQ-£n  y) .  (4) 

Helmbold  also  developed  the  "force-ratio"  equation  and  obtained  a 
solution. 

Let  us  note  that  when  c  =  y,   equation  (3)  yields  a  "linear 
law."  However,  the  reader  should  observe  that  although  equation  (3) 
becomes  a  "linear  law"  when  c  =  y,   equations  (1)  do  not  reduce  to 
Lanchester's  equation  for  area  fire  (see  Appendix  H) ,  and  consequently 
the  time  solution  to  (1)  is  not  the  same  as  that  for  the  classical 
Lanchester  area-fire  equations  (see  [7]  for  the  time  solution). 
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3.  The  Optimal  Control  Problem. 

Motivated  by  Helmbold's  modification  of  Lanchester's  equations, 
we  consider  the  following  optimal  control  problem: 


maximize  {ry(T)  -  px. (T)  -qx_(T)}  with  T  specified, 
♦  (t)  l  2 

dxl  ■  1- 

subject  to:  -^—  =  -  <J>  a^,  y  C, 

dx 

=  -(l-4>)a_x.  y 


dt     v   T/  22 


dy     .   c  '  1-c   ,   c  1-c         ,r. 
dt  =  "  V  xl    -  b2y  X2    •         (5) 


xl'X2,y  ^  °   and   °  **  *  *"  1» 


where 


x  (t) ,x-(t) ,y(t)   are  force  levels, 
p,q,r  are  utilities  assigned  survivors, 
a1 ,a  ,b  ,b_  are  (constant)  attrition-rate  coefficients, 
<J>   is  the  fraction  of  Y  fire  directed  at  X  , 
and   c  is  a  parameter  which  must  satisfy  0  £  c  <  1. 


In  our  initial  investigation  here  we  will  limit  our  discussion  to  the 
development  of  the  basic  necessary  conditions  of  optimality. 
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4 .   Solution  to  Heterogeneous  Combat  Equations. 

It  is  of  interest  to  determine  the  state  solution  to 

dx.  . 

1  c  1-c 


=  -  ^a.x.  y 


dt  r  1  1 

dx 

2  ,■>    x\          c  1-c 

dF~  =  _(H)a2X2  y    ' 

dy  .   c  1-c   ,   c  1-c              ... 

dt  -V  Xl    "  V  x2    '              (6) 

when  <j)  =  constant  for  0  £  t  £  t.  .   Equations  (6)  readily  yield 


zQ2  -  z2(t)  =  {^a^  +  (l-4>)a2b2Hy02  2c  -  (y(t))2  2°},  (7) 


where 


z(t)  =  b1x     +  b2x2 


and 


z(t=0)  -  zQ. 

As  we  have  seen  in  Appendices  A,  F,  and  G,  the  state  solution  (7) 
is  useful  in  synthesizing  optimal  trajectories.   A  similar  simple 
solution  also  arises  in  the  case  of  n-versus-one  combat  according  to 
Helmbold's  general  attrition  model. 

5.  Development  of  Basic  Necessary  Conditions  of  Optimality. 

We  will  focus  on  the  development  of  necessary  conditions  of 
optimality.   It  is  convenient  to  consider  state  variable  inequality 
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constraints  equivalent  to  x  >  0,   x  £  0  for  (5) .   Instead  of 
x  ^  0,  we  consider 

-  x±   "C  £  0, 
and  similarly 

-  x2    s  0, 

where  c  £  [0,1)   so  that  1  -  c  >  0.  When  x  =  0,  we  must  therefore 
have 

d    1-c 
—(-x    )  £  0,   so  that 

(l-c)cj>a  y  ~c  £  0  when  x  =  0,  (8) 

and  similarly 

(l-c)(H)a2y1_C  £  0  when  x2  =  0.         (9) 

Following  Gamkrelidze  (see  Chapter  VI  of  [6]) (see  also  section  4.b.(3) 
of  Appendix  E) ,  the  Hamiltonian  is  given  by 

H(t,xi,pi,<|>)  =  -  p-^a-jy  Cx±C   -  p2(l-4>)a2y  °x2 


-  P3yC(b1x11"C+b2x21-C)  -  y1(t)*a1y1  C  -  y2(t)  (l-^y1  C,    (10) 


where 


M±(t) 


=  0   for  x  >  0, 


^  0   for  x  =  0, 
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and  p   is  the  dual  variable  corresponding  to  x  .   The  adjoint  system 
of  differential  equations  for  the  dual  variables  is 
dp, 


^  =   c^p^-y-  :  +    (l-c)b1P3x1-CyC, 


(ID 


dp2  c-1   1-c  -c 

^-  =  c(l-(t>)a2p2x2        y      -  +    (l-c)b2p3x2     y^ 


(12) 


jf-  =    (l-c)y  C{<j>a1(p1x1C+y1)   +   (l-^a^p^+y^  } 


c-1,,  1-c.,  1-Cv 

+  cp3y        (b^       +b2x2        '  * 


(13) 


When  x  ,x~  >  0,   the  control  low  is  determined  by  the  maximum 
principle.  Hence,  we  consider 


maximize  H(t,x.  ,p  .  ,<J>)  . 
0£4>£l        X  X 


This  yields 


4>*(t)  = 


1  for  a1(-p1(t))x1°  >  a2(-p2(t))x2°, 


0  for  a1(-p1(t))x1°  <  a2(-p2(t))x2°.    (14) 


We  must  further  investigate  the  possibility  of  a  singluar  subarc  [4], 

■ill 

[5]   on  which  —  =  0   for  a  finite  interval  of  time  (see  discussion 
dq> 

in  Appendix  D) . 
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a-   Investigation  of  Singular  Subarcs. 

As  is  well-known,  since  the  Hamiltonian  (10)  is  a  linear  function 
of  the  control  variable  <J>,   the  maximum  principle  does  not  determine 
the  control  when  the  coefficient  of  $   in  H  vanishes  for  a  finite 
interval  of  time  [4],  [5],  On  such  a  singular  subarc  we  must  have 


3H    d  /3H\   rt  ,   v 


3H 
The  condition  —  =  0  yields  that  on  a  singluar  subarc  we  must  have 

alPlXi°  =  a2P2X2°'  (16) 

where  we  have  observed  that  y  >  0.  Combining  (16)  with  the  expression 
f°r  dt(lf)  We  °btain 

£(if)  "  (l-OyP3(a2b2-a1b1)  when  f|  -  0.       (17) 

Assuming  that  c  ^  1  and  anb.  ^  a0b„,  we  see  that  —(77)  =  0 

1±     2.    Z  at  \d(J>  / 

when  —  =  0   implies  that  we  must  have  p~(t)  =  0.  While  we  have 

dtp  J 

obtained  no  contradiction  that  p^(t)  =  0  for  a  finite  interval  of 

time  on  a  singular  subarc,  considering  the  interpretation  of  the 

3S 
dual  variable  p_  =  —  where  S   is  the  optimal  value  function,   it 
J   3y 

intuitively  seems  impossible  that  p^(t)  =  0  for  a  finite  interval 
of  time.   (This  would  imply  that  there  is  no  marginal  return  for 
increasing  the  Y-force  level.)   Thus,  we  tentatively  conclude  that 
there  is  no  singular  solution,  and  the  optimal  policy  is  $*  =   0  or 
1   (with  possible  changes  over  time).   Hence,  we  see  that  for  Helmbold's 
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general  attrition  model  the  optimal  tactic  is  apparently  always  to 
concentrate  all  fire  on  a  single  target  type. 

b.  Optimality  Conditions  for  Force  Level  to  be  Driven  to  Zero. 

Let  us  now  investigate  whether  it  is  a  good  policy  to  annihilate 
a  force  type  in  this  model.   (The  reader  can  find  a  discussion  of  the 
theory  of  state  variable  inequality  constraints  that  we  apply  here  in 
Appendix  E.)   Without  loss  of  generality,  we  may  consider  a  constrained 
subarc  on  which  x..  (t)  =  0   (and  x~  >  0)   for  t..  £  t  £  T.   The  control 
is  clearly  <J>*(t)  =  0.   The  multiplier  y  (t)   is  determined  by  the 
condition  —  =  0  and  hence 

U-^t)  =  "^"("Pi3]^!  +P2a2x2  ^'  ^18^ 

Differentiating  (19)  and  combining  with  (5) ,  (11) ,  and  (12) ,  we  find 
that 

;1(t)---a^-p37c(^b1^2fc2),  (i9) 

so  that  Gamkrelidze's  condition  y,  (t)  £  0   [6]  is  only  satisfied  on 
a  constrained  subarc  with  x  =  0  when 

albl  *  a2b2  (20) 

since  we  would  expect  to  have  p~(t)  ^  0. 

However,  even  if  (20)  is  satisfied,  it  is  never  an  optimal  policy 
to  drive  x..   to  zero  (unlike  the  corresponding  situation  for  the 
Isbell-Marlow  problem  discussed  in  Section  4.b.(l)  of  Appendix  E) . 
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The  proof  of  this  statement  is  by  contradiction.  The  necessary 
condition  that  y^t)  ;>  0  where  y   is  given  by  (18)  combined  with 
x  =  0  yields  that 

P2(t)  ^0  for   t-  *  t  s6  T.  (21) 

However,  for  x  (T)  >  0  we  have  the  boundary  condition 

P2(t=T)  =  -  q.  (22) 

By  the  inconsistency  of  (21)  and  (22) (both  of  which  are  necessary 
conditions  of  optimality)  it  can  never  be  an  optimal  policy  to  drive 
x   to  zero  (and  have  x  >  0) . 

6.   Discussion. 

The  results  presented  in  this  appendix  are  preliminary,  being 
based  on  an  initial  cursory  examination.  We  feel  that  this  is  an 
important  and  promising  area  and  propose  this  to  ONR  as  a  future 
research  task. 

We  tentatively  concluded  that  the  optimal  policy  was  always  to 
concentrate  all  fire  on  one  target  type  (the  same  optimal  policy  as 
for  the  one-versus-n  problem  (see  Appendix  C)  with  a  "square-law 
attrition"  of  target  types).   In  other  words,  our  analysis  so  far  has 
disclosed  no  singular  solution  [5]  to  (5) .   Considering  the  time 
solution  given  by  (2)  to  the  combat  equations  (1) ,  we  see  (heuristically) 
that  the  time  solution  to  equations  (1)  when  c  =  —  is  closer  in  form 
to  that  for  a  "square-law"  battle  than  that  for  Lanchester's  classical 
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area-fire  equations  [7].   Hence,  considering  the  results  of  Appendix  D, 
our  tentative  conclusion  here  on  concentration  of  fire  as  an  optimal 
policy  is  intuitively  appealing,  although  we  feel  that  more  work  remains 
to  be  done. 

We  finally  saw  that  if  x  >  0,   then  it  was  not  an  optimal 
policy  to  drive  x   to  zero.   Considering  (14),  this  makes  sense, 
since  as  x-   goes  to  zero  we  eventually  must  have  a1(-p1(t))x1   < 
a_(-p_(t))x-  .   Intuitively  this  policy  also  makes  sense,  since  the 
motivation  behind  Helmbold's  formulation  (1)  is  inefficiencies  of 
scale  for  the  larger  force.   Hence,  as  x..   "becomes  small  enough" 
the  Y-force  is  more  efficient  at  destroying  X». 
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